BITC 421 Multimedia Systems- Complementary Reading

Topics
1. Multimedia Workstation
2. Basic Tools for Multimedia Objects
3. User Interface
4. Multimedia Communication Systems

1. Multimedia Workstation
Introduction
A multimedia workstation is computer with facilities to handle multimedia objects such as text,
audio, video, animation and images. A multimedia workstation was earlier identified as MPC
(Multimedia Personal Computer). In the current scenario all computers are prebuilt with
multimedia processing facilities. Hence it is not necessary to identify a computer as MPC.A
multimedia system is comprised of both hardware and software components, but the major driving
force behind a multimedia development is research and development in hardware capabilities.
Besides the multimedia hardware capabilities of current personal computers (PCs) and
workstations, computer networks with their increasing throughput and speed start to offer services
which support multimedia communication systems. Also in this area, computer networking
technology advances faster than the software.
Communication Architecture
Local multimedia systems (i.e., multimedia workstations) frequently include a network interface
(e.g., Ethernet card) through which they can communicate with each other. However, the
transmission of audio and video cannot be carried out with only the conventional communication
infrastructure and network adapters. Until now, the solution was that continuous and discrete
media have been considered in different environments, independently of each other. It means that
fully different systems were built. For example, on the one hand, the analog telephone system
provides audio transmission services using its original dial devices connected by copper wires to
the telephone company’s nearest end office. The end offices are connected to switching centers,
called toll offices, and these centers are connected through high bandwidth inter-toll trunks to
intermediate switching offices. This hierarchical structure allows for reliable audio
communication. On the other hand, digital computer networks provide data transmission services
at lower data rates using network adapters connected by copper wires to switches and routers.
Even today, professional radio and television studios transmit audio and video streams in the form
of analog signals, although most network components (e.g., switches), over which these signals
are transmitted, work internally in a digital mode.
Hybrid Systems
By using existing technologies, integration and interaction between analog and digital
environments can be implemented. This integration approach is called the hybrid approach.
The main advantage of this approach is the high quality of audio and video and all the necessary
devices for input, output, storage and transfer that are available. The hybrid approach is used for
studying application user interfaces, application programming interfaces or application scenarios.
Integrated Device Control
One possible integration approach is to provide a control of analog input/output audio-video
components in the digital environment. Moreover, the connection between the sources (e.g., CD
player, camera, microphone) and destinations (e.g., video recorder, write-able CD), or the
switching of audio-video signals can be controlled digitally.
Integrated Transmission Control
A second possibility to integrate digital and analog components is to provide a common
transmission control. This approach implies that analog audio-video sources and destinations are
connected to the computer for control purposes to transmit continuous data over digital networks,
such as a cable network.
Integrated Transmission
The next possibility to integrated digital and analog components is to provide a common
transmission network. This implies that external analog audio-video devices are connected to
computers using A/D (D/A) converters outside of the computer, not only for control, but also for
processing purposes. Continuous data are transmitted over shared data networks.
Digital Systems
Connection to Workstations
In digital systems, audio-video devices can be connected directly to the computers (workstations)
and digitized audio-video data are transmitted over shared data networks, Audio-video devices in
these systems can be either analog or digital.
Connection to switches
Another possibility to connect audio-video devices to a digital network is to connect them directly
to the network switches.
Multimedia Workstation
Current workstations are designed for the manipulation of discrete media information. The data
should be exchanged as quickly as possible between the involved components, often
interconnected by a common bus. Computationally intensive and dedicated processing
requirements lead to dedicated hardware, firmware and additional boards.
Examples of these components are hard disk controllers and FDDI-adapters. A multimedia
workstation is designed for the simultaneous manipulation of discrete and continuous media
information. The main components of a multimedia workstation are:
✓ Standard Processor(s) for the processing of discrete media information.
✓ Main Memory and Secondary Storage with corresponding autonomous
controllers.
✓ Universal Processor(s) for processing of data in real-time (signal processors).
✓ Special-Purpose Processors designed for graphics, audio and video media
(containing, for example, a micro code decompression method for DVI
processors).
✓ Graphics and video Adapters.
✓ Communications Adapters (for example, the Asynchronous Transfer Mode
Host Interface.
✓ Further special-purpose adapters.
Bus
Within current workstations, data are transmitted over the traditional asynchronous bus, meaning
that if audio-video devices are connected to a workstation, continuous data are processed in a
workstation, and the data transfer is done over this bus, which provides low and unpredictable time
guarantees. In multimedia workstations, in addition to this bus, the data will be transmitted over a
second bus which can keep time guarantees. In later technical implementations, a bus may be
developed which transmits two kinds of data according to their requirements (this is known as a
multi-bus system).
The notion of a bus has to be divided into system bus and periphery bus. In their current versions,
system busses such as ISA, EISA, Microchannel, Q-bus and VME-bus support only limited
transfer of continuous data. The further development of periphery busses, such as SCSI, is aimed
at the development of data transfer for continuous media.
Multimedia Devices
The main peripheral components are the necessary input and output multimedia devices. Most of
these devices were developed for or by consumer electronics, resulting in the relative low cost of
the devices. Microphones, headphones, as well as passive and active speakers, are examples. For
the most part, active speakers and headphones are connected to the computer because it, generally,
does not contain an amplifier. The camera for video input is also taken from consumer electronics.
Hence, a video interface in a computer must accommodate the most commonly used video
techniques/standards, i.e., NTSC, PAL, SECAM with FBAS, RGB, YUV and YIQ modes. A
monitor serves for video output. Besides Cathode Ray Tube (CRT) monitors (e.g., current
workstation terminals), more and more terminals use the color-LCD technique (e.g., a projection
TV monitor uses the LCD technique). Further, to display video, monitor characteristics, such as
color, high resolution, and flat and large shape, are important.
Primary Storage
Audio and video data are copied among different system components in a digital system. An
example of tasks, where copying of data is necessary, is a segmentation of the LDUs or the
appending of a Header and Trailer. The copying operation uses system software-specific memory
management designed for continuous media. This kind of memory management needs sufficient
main memory (primary storage). Besides ROMs, PROMs, EPROMS, and partially static memory
elements, low-cost of these modules, together with steadily increasing storage capacities, profits
the multimedia world.
Secondary Storage
The main requirements put on secondary storage and the corresponding controller is a high storage
density and low access time, respectively.
On the one hand, to achieve a high storage density, for example, a Constant Linear Velocity (CLV)
technique was defined for the CD-DA (Compact Disc Digital Audio). CLV guarantees that the
data density is kept constant for the entire optical disk at the expense of a higher mean access time.
On the other hand, to achieve time guarantees, i.e., lower mean access time, a Constant Angle
Velocity (CAV) technique could be used.
Because the time requirement is more important, the systems with a CAV are more
suitable for multimedia than the systems with a CLV.
Processor
In a multimedia workstation, the necessary work is distributed among different processors.
Although currently, and for the near future, this does not mean that all multimedia workstations
must be multi-processor systems. The processors are designed for different tasks. For example, a
Dedicated Signal Processor (DSP) allows compression and decompression of audio in real-time.
Moreover, there can be special-purpose processors employed for video. The following Figure
shows an example of a multiprocessor for multimedia workstations envisioned for the future.
Operating System
Another possible variant to provide computation of discrete and continuous data in a multimedia
workstation could be distinguishing between processes for discrete data computation and for
continuous data processing. These processes could run on separate processors. Given an adequate
operating system, perhaps even one processor could be shared according to the requirements
between processes for discrete and continuous data.
Preference of Operating System for Workstation.
Selection of the proper platform for developing the multimedia project may be based on your
personal preference of computer, your budget constraints, and project delivery requirements, and
the type of material and content in the project. Many developers believe that multimedia project
development is smoother and easier on the Macintosh than in Windows, even though projects
destined to run in Windows must then be ported across platforms. But hardware and authoring
software tools for Windows have improved; today you can produce many multimedia projects with
equal ease in either the Windows or Macintosh environment.
The Macintosh Platform
All Macintoshes can record and play sound. Many include hardware and software for digitizing
and editing video and producing DVD discs. High-quality graphics capability is available “out of
the box.” Unlike the Windows environment, where users can operate any application with
keyboard input, the Macintosh requires a mouse.
The Macintosh computer you will need for developing a project depends entirely upon the
project’s delivery requirements, its content, and the tools you will need for production.
The Windows Platform
Unlike the Apple Macintosh computer, a Windows computer is not a computer per se, but rather
a collection of parts that are tied together by the requirements of the Windows operating system.
Power supplies, processors, hard disks, CD-ROM players, video and audio components, monitors,
key-boards and mice-it doesn’t matter where they come from or who makes them. Made in Texas,
Taiwan, Indonesia, Ireland, Mexico, or Malaysia by widely known or little-known manufactures,
these components are assembled and branded by Dell, IBM, Gateway, and other into computers
that run Windows.
In the early days, Microsoft organized the major PC hardware manufactures into the Multimedia
PC Marketing Council to develop a set of specifications that would allow Windows to deliver a
dependable multimedia experience.
Networking Macintosh and Windows Computers
When a user works in a multimedia development environment consisting of a mixture of
Macintosh and Windows computers, you will want them to communicate with each other. It may
also be necessary to share other resources among them, such as printers.
Local area networks (LANs) and wide area networks (WANs) can connect the members of a
workgroup. In a LAN, workstations are usually located within a short distance of one another, on
the same floor of a building, for example. WANs are communication systems spanning great
distances, typically set up and managed by large corporation and institutions for their own use, or
to share with other users. LANs allow direct communication and sharing of peripheral resources
such as file servers, printers, scanners, and network modems. They use a variety of proprietary
technologies, most commonly Ethernet or TokenRing, to perform the connections. They can
usually be set up with twisted-pair telephone wire, but be sure to use “data-gradeleve l 5” or “cat-
5” wire-it makes a real difference, even if it’s a little more expensive! Bad wiring will give the
user never-ending headache of intermittent and often untraceable crashes and failures.
2. Basic Tools for Multimedia Objects
Introduction
The basic tools set for building multimedia project contains one or more authoring systems and
various editing applications for text, images, sound, and motion video. A few additional
applications are also useful for capturing images from the screen, translating file formats and tools
for the making multimedia production easier.
Text Editing and Word Processing Tools
A word processor is usually the first software tool computer users rely upon for creating text. The
word processor is often bundled with an office suite. Word processors such as Microsoft Word
and WordPerfect are powerful applications that include spellcheckers, table formatters,
thesauruses and prebuilt templates for letters, resumes, purchase orders and other common
documents.
OCR Software
Often there will be multimedia content and other text to incorporate into a multimedia project, but
no electronic text file. With optical character recognition (OCR) software, a flat-bed scanner, and
a computer, it is possible to save many hours of rekeying printed words, and get the job done faster
and more accurately than a roomful of typists. OCR software turns bitmapped characters into
electronically recognizable ASCII text. A scanner is typically used to create the bitmap. Then the
software breaks the bitmap into chunks according to whether it contains text or graphics, by
examining the texture and density of areas of the bitmap and by detecting edges. The text areas of
the image are then converted to ASCII character using probability and expert system algorithms.
Image-Editing Tools
Image-editing application is specialized and powerful tools for enhancing and retouching existing
bitmapped images. These applications also provide many of the feature and tools of painting and
drawing programs and can be used to create images from scratch as well as images digitized from
scanners, video frame-grabbers, digital cameras, clip art files, or original artwork files created with
a painting or drawing package.
Here are some features typical of image-editing applications and of interest to
multimedia developers:
Multiple windows that provide views of more than one image at a time
Conversion of major image-data types and industry-standard file formats
Direct inputs of images from scanner and video sources
Employment of a virtual memory scheme that uses hard disk space as RAM for images that
require large amounts of memory
Capable selection tools, such as rectangles, lassos, and magic wands, to select portions of a
bitmap
Image and balance controls for brightness, contrast, and color balance
Good masking features
Multiple undo and restore features
Anti-aliasing capability, and sharpening and smoothing controls
Color-mapping controls for precise adjustment of color balance
Tools for retouching, blurring, sharpening, lightening, darkening, smudging, and tinting
Geometric transformation such as flip, skew, rotate, and distort, and perspective changes
Ability to resample and resize an image
134-bit color, 8- or 4-bit indexed color, 8-bit gray-scale, black-and-white, and customizable
color palettes
 Ability to create images from scratch, using line, rectangle, square, circle, ellipse, polygon,
airbrush, paintbrush, pencil, and eraser tools, with customizable brush shapes and user-definable
bucket and gradient fills
Multiple typefaces, styles, and sizes, and type manipulation and masking routines
Filters for special effects, such as crystallize, dry brush, emboss, facet, fresco, graphic pen,
mosaic, pixelize, poster, ripple, smooth, splatter, stucco, twirl, watercolor, wave, and wind
Support for third-party special effect plug-ins
Ability to design in layers that can be combined, hidden, and reordered
Plug-Ins
Image-editing programs usually support powerful plug-in modules available from third-party
developers that allow to wrap, twist, shadow, cut, diffuse, and otherwise “filter” your images for
special visual effects.

Painting and Drawing Tools

Painting and drawing tools, as well as 3-D modelers, are perhaps the most important items in the
toolkit because, of all the multimedia elements, the graphical impact of the project will likely have
the greatest influence on the end user. If the artwork is amateurish, or flat and uninteresting, both
the creator and the users will be disappointed.
Painting software, such as Photoshop, Fireworks, and Painter, is dedicated to producing crafted
bitmap images. Drawing software, such as CorelDraw, FreeHand, Illustrator, Designer, and
Canvas, is dedicated to producing vector-based line art easily printed to paper at high resolution.
Some software applications combine drawing and painting capabilities, but many authoring
systems can import only bitmapped images. Typically, bitmapped images provide the greatest
choice and power to the artist for rendering fine detail and effects, and today bitmaps are used in
multimedia more often than drawn objects. Some vector based packages such as Macromedia’s
Flash are aimed at reducing file download times on the Web, and may contain both bitmaps and
drawn art. The anti-aliased character shown in the bitmap of Color Plate 5 is an example of the
fine touches that improve the look of an image.
Look for these features in a drawing or painting packages:
An intuitive graphical user interface with pull-down menus, status bars, palette control, and
dialog boxes for quick, logical selection
Scalable dimensions, so you can resize, stretch, and distort both large and small bitmaps
Paint tools to create geometric shapes, from squares to circles and from curves to complex
polygons
Ability to pour a color, pattern, or gradient into any area
Ability to paint with patterns and clip art
Customizable pen and brush shapes and sizes
Eyedropper tool that samples colors
Auto trace tool that turns bitmap shapes into vector-based outlines
Support for scalable text fonts and drop shadows
Multiple undo capabilities, to let you try again
Painting features such as smoothing coarse-edged objects into the background with anti-aliasing,
airbrushing in variable sizes, shapes, densities, and patterns; washing colors in gradients; blending;
and masking
Support for third-party special effect plug-ins
Object and layering capabilities that allow you to treat separate elements independently
 Zooming, for magnified pixel editing
All common color depths: 1-, 4-, 8-, and 16-, 134-, or 313- bit color, and grayscale
Good color management and dithering capability among color depths using
various color models such as RGB, HSB, and CMYK
Good palette management when in 8-bit mode
Good file importing and exporting capability for image formats such as PIC, GIF,
TGA, TIF, WMF, JPG, PCX, EPS, PTN, and BMP
Sound Editing Tools
Sound editing tools for both digitized and MIDI sound lets hear music as well as create it. By
drawing a representation of a sound in fine increments, whether a score or a waveform, it is
possible to cut, copy, paste and otherwise edit segments of it with great precision. System sounds
are shipped both Macintosh and Windows systems and they are available as soon the Operating
system is installed. For MIDI sound, a MIDI synthesizer is required to play and record sounds
from musical instruments. For ordinary sound there are varieties of software such as Soundedit,
MP3cutter, Wavestudio.
Animation, Video and Digital Movie Tools
Animation and digital movies are sequences of bitmapped graphic scenes (frames, rapidly played
back. Most authoring tools adapt either a frame or object oriented approach to animation.
Moviemaking tools typically take advantage of Quicktime for Macintosh and Microsoft Video for
Windows and lets the content developer to create, edit and present digitized motion video
segments.
Video formats
A video format describes how one device sends video pictures to another device, such as the way
that a DVD player sends pictures to a television or a computer to a monitor. More formally, the
video format describes the sequence and structure of frames that create the moving video image.
Video formats are commonly known in the domain of commercial broadcast and consumer
devices; most notably to date, these are the analog video formats of NTSC, PAL, and SECAM.
However, video formats also describe the digital equivalents of the commercial formats, the aging
custom military uses of analog video (such as RS-170 and RS-343), the increasingly important
video formats used with computers, and even such offbeat formats such as color field sequential.
Video formats were originally designed for display devices such as CRTs. However, because other
kinds of displays have common source material and because video formats enjoy wide adoption
and have convenient organization, video formats are a common means to describe the structure of
displayed visual information for a variety of graphical output devices.
Common organization of video formats
A video format describes a rectangular image carried within an envelope containing information
about the image. Although video formats vary greatly in organization, there is a common
taxonomy:
A frame can consist of two or more fields, sent sequentially, that are displayed over time to form
a complete frame. This kind of assembly is known as interlace. An interlaced video frame is
distinguished from a progressive scan frame, where the entire frame is sent as a single intact entity.
A frame consists of a series of lines, known as scan lines. Scan lines have a regular and consistent
length in order to produce a rectangular image. This is because in analog formats, a line lasts for a
given period of time; in digital formats, the line consists of a given number of pixels. When a
device sends a frame, the video format specifies that devices send each line independently from
any others and that all lines are sent in top-to-bottom order.
 As above, a frame may be split into fields – odd and even (by line "numbers") or upper and
lower, respectively. In NTSC, the lower field comes first, then the upper field, and that's the whole
frame. The basics of a format are Aspect Ratio, Frame Rate, and Interlacing with field order if
applicable: Video formats use a sequence of frames in a specified order. In some formats, a single
frame is independent of any other (such as those used in computer video formats), so the
sequence is only one frame. In other video formats, frames have an ordered position. Individual
frames within a sequence typically have similar construction.

However, depending on its position in the sequence, frames may vary small elements within them
to represent additional information. For example, MPEG-13 compression may eliminate the
information that is redundant frame-to-frame in order to reduce the data size, preserving the
information relating to changes between frames.

Analog video formats

NTSC
PAL
SECAM
Digital Video Formats
These are MPEG13 based terrestrial broadcast video formats
ATSC Standards
DVB
ISDB
These are strictly the format of the video itself, and not for the modulation used for transmission.
QuickTime
QuickTime is a multimedia framework developed by Apple Inc. capable of handling various
formats of digital video, media clips, sound, text, animation, music, and several types of interactive
panoramic images. Available for Classic Mac OS, Mac OS X and Microsoft Windows operating
systems, it provides essential support for software packages including iTunes, QuickTime Player
(which can also serve as a helper application for web browsers to play media files that might
otherwise fail to open) and Safari.
The QuickTime technology consists of the following:
1. The QuickTime Player application created by Apple, which is a media player.
2. The QuickTime framework, which provides a common set of APIs for encoding
and decoding audio and video.
3. The QuickTime Movie (.mov) file format, an openly-documented media
container.
QuickTime is integral to Mac OS X, as it was with earlier versions of Mac OS. All Apple systems
ship with QuickTime already installed, as it represents the core media framework for Mac OS X.
QuickTime is optional for Windows systems, although many software applications require it.
Apple bundles it with each iTunes for Windows download, but it is also available as a stand-alone
installation.
QuickTime players
QuickTime is distributed free of charge, and includes the QuickTime Player application. Some
other free player applications that rely on the QuickTime framework provide features not available
in the basic QuickTime Player. For example:
iTunes can export audio in WAV, AIFF, MP3, AAC, and Apple Lossless.
 In Mac OS X, a simple AppleScript can be used to play a movie in full-screen mode. However,
since version 7.13 the QuickTime Player now also supports for full screen viewing in the non-pro
version.
QuickTime framework
The QuickTime framework provides the following:
Encoding and transcoding video and audio from one format to another.
Decoding video and audio, and then sending the decoded stream to the graphics or audio
subsystem for playback. In Mac OS X, QuickTime sends video playback to the Quartz Extreme
(OpenGL) Compositor.
A plug-in architecture for supporting additional codecs (such as DivX).
The framework supports the following file types and codecs natively:
Audio
Apple Lossless
Audio Interchange (AIFF)
Digital Audio: Audio CD - 16-bit (CDDA), 134-bit, 313-bit integer & floating point, and 64-bit
floating point
MIDI
MPEG-1 Layer 3 Audio (.mp3)
MPEG-4 AAC Audio (.m4a, .m4b, .m4p)
Sun AU Audio
ULAW and ALAW Audio
Waveform Audio (WAV)
Video
3GPP & 3GPP13 file formats
AVI file format
Bitmap (BMP) codec and file format
DV file (DV NTSC/PAL and DVC Pro NTSC/PAL codecs)
Flash & FlashPix files
GIF and Animated GIF files
H.1361, H.1363, and H.1364 codecs
JPEG, Photo JPEG, and JPEG-13000 codecs and file formats
MPEG-1, MPEG-13, and MPEG-4 Video file formats and associated codecs
(such as AVC)
QuickTime Movie (.mov) and QTVR movies
Other video codecs: Apple Video, Cinepak, Component Video, Graphics, and
Planar RGB
Other still image formats: PNG, TIFF, and TGA
Specification for QuickTime file format
QuickTime Movie
File extension: .mov .qt
MIME type: video/quicktime
Type code: MooV
Uniform Type Identifier: [Link]-movie
Developed by: Apple Inc.
Type of format: Media container
Container for: Audio, video, text
The QuickTime (.mov) file format functions as a multimedia container file that contains
one or more tracks, each of which stores a particular type of data: audio, video, effects, or
text (for subtitles, for example). Other file formats that QuickTime supports natively (to
varying degrees) include AIFF, WAV, DV, MP3, and MPEG-1. With additional
QuickTime Extensions, it can also support Ogg, ASF, FLV, MKV, DivX Media Format,
and others.

3. User Interface
Introduction
In computer science, we understand the user interface as the interactive input and output of a
computer as it’s is perceived and operated on by users. Multimedia user interfaces are used for
making the multimedia content active. Without user interface the multimedia content is considered
to be linear or passive.
User Interfaces
Multimedia user interfaces are computer interfaces that communicate with users using multiple
media modes such as written text together with spoken language. Multimedia would be without
much value without user interfaces. The input media determines not only how human computer
interaction occurs but also how well. Graphical user interfaces – using the mouse as the main input
device – have greatly simplified human-machine interaction.
General Design Issues
The main emphasis in the design of multimedia user interface is multimedia presentation. There
are several issues which must be considered.
1. To determine the appropriate information content to be communicated.
2. To represent the essential characteristics of the information.
3. To represent the communicative intent.
4. To chose the proper media for information presentation.
5. To coordinate different media and assembling techniques within a presentation.
6. To provide interactive exploration of the information presented.
Information Characteristics for presentation:
A complete set of information characteristics makes knowledge definitions and representation
easier because it allows for appropriate mapping between information andbpresentation
techniques. The information characteristics specify:
Types
Characterization schemes are based on ordering information. There are two types of bordered data:
(1) coordinate versus amount, which signify points in time, space or other domains; or (2) intervals
versus ratio, which suggests the types of comparisons meaningful among elements of coordinate
and amount data types.
Relational Structures
This group of characteristics refers to the way in which a relation maps among its domain sets
(dependency). There are functional dependencies and non-functional dependencies. An example
of a relational structure which expresses functional dependency is a bar chart. An example of a
relational structure which expresses nonfunctional dependency is a student entry in a relational
database.
Multi-domain Relations
Relations can be considered across multiple domains, such as:
(1) multiple attributes of a single object set (e.g., positions, colors, shapes, and/or sizes of a set of
objects in a chart);
(2) multiple object sets (e.g., a cluster of text and graphical symbols on a map); and
(3) multiple displays.
Large Data Sets
Large data sets refer to numerous attributes of collections of heterogeneous objects (e.g.,
presentations of semantic networks, databases with numerous object types and attributes of
technical documents for large systems, etc.).
Presentation Function
Presentation function is a program which displays an object. It is important to specify the
presentation function independent from presentation form, style or the information it conveys.
Several approaches consider the presentation function from different points of view
15.3.3 Presentation Design Knowledge
To design a presentation, issues like content selection, media and presentation technique selection
and presentation coordinating must be considered. Control selection is the key to convey the
information to the user. However, we are not free in the selection of it because content can be
influenced by constraints imposed by the size and complexity of the presentation. Media selection
determines partly the information characteristics. For selecting presentation techniques, rule can
be used. For example, rules for selection methods, i.e., for supporting a users ability to locate on
the facts in a presentation, may specify a preference for graphical techniques. Coordination can be
viewed as a process of composition. Coordination needs mechanisms such as
(1) encoding techniques
(2) presentation objects that represent facts
(3) multiple displays. Coordination of multimedia employs a set of composition operators for
merging, aligning and synthesizing different objects to construct displays that convey multiple
attributes of one or more data sets.
Effective Human-Computer Interaction
One of the most important issues regarding multimedia interfaces is effective human-computer
interaction of the interface, i.e., user-friendliness.
The main issues the user interface designer should keep in mind:
(1) context;
(2) linkage to the world beyond the presentation display;
(3) evaluation of the interface with respect to other human-computer interfaces;
(4) interactive capabilities; and
(5) separability of the user interface from the application.
Video at the User Interface
A continuous sequence of, at least, 15 individual image per second gives a rough perception of a
continuous motion picture. At the use interface, video is implemented through a continuous
sequence of individual images. Hence, video can be manipulated at this interface similar to
manipulation of individual still images. When an individual image consisting of pixels (no
graphics, consisting of defined objects) can be presented and modified, this should also be possible
for video (e.g., to create special effects in a movie). However, the functionalities for video are not
as simple to deliver because the high data transfer rate necessary is not guaranteed by most of the
hardware in current graphics systems.
Audio at the User Interface
Audio can be implemented at the user interface for application control. Thus, speech analysis is
necessary. Speech analysis is either speaker-dependent or speaker-independent. Speakerdependent
solutions allow the input of approximately 25,000 different words with a relatively low error rate.
Here, an intensive learning phase to train the speech analysis system for speaker-specific
characteristics is necessary prior to the speech analysis phase.
A speaker-independent system can recognize only a limited set of words and no training
phase is needed.
Audio Tool User Interface
During audio output, the additional presentation dimension of space can be introduced using two
or more separate channels to give a more natural distribution of sound. The best-known example
of this technique is stereo.
In the case of monophony, all audio sources have the same spatial location. A
listener can only properly understand the loudest audio signal. The same effect can be
simulated by closing one ear. Stereophony allows listeners with bilateral hearing capabilities to
hear lower intensity sounds. It is important to mention that the main advantage of bilateral hearing
is not the spatial localization of audio sources, but the extraction of less intensive signals in a loud
environment.
User-friendliness as the Primary Goal
User-friendliness is the main property of a good user interface. In a multimedia environment in
office or home the following user friendliness could be implemented:
Easy to Learn Instructions
Application instructions must be easy-to-learn.
Context-sensitive Help Functions
A context-sensitive help function using hypermedia techniques is very helpful, I e., according to
the state of the application, different help-texts are displayed.
Easy to Remember Instructions
A user-friendly interface must also have the property that the user easily
remembers the application instruction rules. Easily remembered instructions
might be supported by the intuitive association to what the user already knows.
Effective Instructions
The user interface should enable effective use of the application. This means:
Logically connected functions should be presented together and similarly.
Graphically symbols or short clips are more effective than textual input and output. They trigger
faster recognition.
Different media should be able to be exchanged among different applications.
Actions should be activated quickly.
A configuration of a user interface should be usable by both professional and sporadic users.
Aesthetics
With respect to aesthetics, the color combination, character sets, resolution and form of the window
need to be considered. They determine a user’s first and lasting
Entry elements
User interfaces use different ways to specify entries for the user:
Entries in a menu
In menus there are visible and non-visible entry elements. Entries which are relevant task are to be
made available for east menu selection
Entries on a graphical interface
 If the interface includes text, the entries can be marked through color and/or different font
If the interface includes images, the entries can be written over the image.
Presentation
The presentation, i.e., the optical image at the user interface, can have the following variants:
Full text
Abbreviated text
Icons, i.e., graphics
Micons, i.e., motion video
Dialogue Boxes
Different dialogue boxes should have similar constructions. This requirement
applies to the design of:
(1) The buttons OK and Abort;
(2) Joined windows; and
(3) Other applications in the same window system.
Additional Design Criteria
A few additional hints for designing a user friendly interface are
The form of the cursor can change to visualize the current state of the system.
For example a hour glass can be shown for a processing task in progress
When time intensive tasks are performed, the progress of the task should be
presented.
The selected entry should be immediately highlights as “work in progress”
before performance actually starts.
The main emphasis has been on video and audio media because they represent live information.
At the user interface, these media become important because they help users learn by enabling
them to choose how to distribute research responsibilities among applications (e.g., on-line
encyclopedias, tutors, simulations), to compose and integrate results and to share learned material
with colleagues (e.g., video conferencing). Additionally, computer applications can effectively do
less reasoning about selection of a multimedia element (e.g., text, graphics, animation or sound)
since alternative media can be selected by the user.

4. Multimedia Communication Systems

Introduction
The consideration of multimedia applications supports the view that local systems expand toward
distributed solutions. Applications such as kiosks, multimedia mail, collaborative work systems,
virtual reality applications and others require high-speed networks with a high transfer rate and
communication systems with adaptive, lightweight transmission protocols on top of the networks.
From the communication perspective, we divide the higher layers of the Multimedia
Communication System (MCS) into two architectural subsystems: an application subsystem and a
transport subsystem.
Application Subsystem
Collaborative Computing
The current infrastructure of networked workstations and PCs, and the availability of audio and
video at these end-points, makes it easier for people to cooperate and bridge space and time. In
this way, network connectivity and end-point integration of multimedia provides users with a
collaborative computing environment. Collaborative computing is generally known as Computer-
Supported Cooperative Work (CSCW).
There are many tools for collaborative computing, such as electronic mail, bulletin boards (e.g.,
Usenet news), screen sharing tools (e.g., ShowMe from Sunsoft), text-based conferencing
systems(e.g., Internet Relay Chat, CompuServe, American Online), telephone conference systems,
conference rooms (e.g., VideoWindow from Bellcore), and video conference systems (e.g.,
MBone tools nv, vat). Further, there are many implemented CSCWsystems that unify several tools,
such as Rapport from AT&T, MERMAID from NEC and others.
Collaborative Dimensions
Electronic collaboration can be categorized according to three main parameters: time, user scale
and control. Therefore, the collaboration space can be partitioned into a three-dimensional space.
Time
With respect to time, there are two modes of cooperative work: asynchronous and synchronous.
Asynchronous cooperative work specifies processing activities that do not
happen at the same time; the synchronous cooperative work happens at the same time.
User Scale
The user scale parameter specifies whether a single user collaborates with another user or a group
of more that two users collaborate together. Groups can be further classified as follows:
A group may be static or dynamic during its lifetime. A group is static if its participating
members are pre-determined and membership does not change during the activity. A group is
dynamic if the number of group members varies during the collaborative activity, i.e., group
members can join or leave the activity at any time.
Group members may have different roles in the CSCW, e.g., a member of a group (if he or she
is listed in the group definition), a participant of a group activity (if he or she successfully joins
the conference), a conference initiator, a conferencechairman, a token holder or an observer.
Groups may consist of members who have homogeneous or heterogeneous or heterogeneous
characteristics and requirements of their collaborative environment.
Control
Control during collaboration can be centralized or distributed. Centralized control means that there
is a chairman (e.g., main manger) who controls the collaborative work and every group member
(e.g., user agent) reports to him or her. Distributed control means that every group member has
control over his/her own tasks in the collaborative work and distributed control protocols are in
place to provide consistent collaboration. Other partition parameter may include locality, and
collaboration awareness. Locality partition means that a collaboration can occur either in the same
place (e.g., a group meeting in an officer or conference room) or among users located in different
place through tele-collaboration.
Group communication systems can be further categorized into computer augmented
collaboration systems, where collaboration is emphasized, and collaboration augmented
computing systems, where the concentrations are on computing.
Group Communication Architecture
Group communication (GC) involves the communication of multiple users in a synchronous or an
asynchronous mode with centralized or distributed control. A group communication architecture
consists of a support model, system model and interface model. The GC support model includes
group communication agents that communicate via a multi-point multicast communication
network
Group communication agents may use the following for their collaboration:
 Group Rendezvous
Group rendezvous denotes a method which allows one to organize meetings, and to get information
about the group, ongoing meetings and other static and dynamic information.
Shared Applications
Application sharing denotes techniques which allow one to replicate information to multiple users
simultaneously. The remote users may point
to interesting aspects (e.g., via tele-pointing) of the information and modify it so that all users can
immediately see the updated information (e.g., joint editing). Shared applications mostly belong
to collaboration transparent applications.
Conferencing
Conferencing is a simple form of collaborative computing. This service provides the management
of multiple users for communicating with each other using multiple media. Conferencing
applications belong to collaboration-aware applications.
The GC system model is based on a client-server model. Clients provide user interfaces for smooth
interaction between group members and the system. Servers supply functions for accomplishing
the group communication work, and each server specializes in its own function.
Communication (Transport) Support conferencing
Group Group Communication Agent
Rendezvous Communication (Transport) Support
Application Group
Sharing Rendezvous
conferencing Application
Group Communication Agent Sharing
Communication (Transport) Support conferencing
Group Group Multicast Communication Agent
Rendezvous Communication
Application Network
Sharing
Group communication support model
Application Sharing Approach
Sharing applications is recognized as a vital mechanism for supporting group communication
activities. Sharing applications means that when a shared application program (e.g., editor)
executes any input from a participant, all execution results performed on the shared object (e.g.,
document text) are distributed among all the participants. Shared objects are displayed, generally,
in shared windows.
Application sharing is most often implemented in collaboration-transparent systems, but can also
be developed through collaboration-aware, special-purpose applications. An example of a software
toolkit that assists in development of shared computer applications is Bell core’s Rendezvous
system (language and architecture). Shared applications may be used as conversational props in
tele-conferencing situations for collaborative document editing and collaborative software
development. An important issue in application sharing is shared control. The primary design
decision in sharing applications is to determine whether they should be centralized or replicated:
Centralized Architecture In a centralized architecture, a single copy of the shared application
runs at one site. All participants’ input to the application is then distributed to all sites. The
advantage of the centralized approach is easy maintenance because there isonly one copy of the
application that updates the shared object. The disadvantage is high network traffic because the
output of the application needs to be distributed every time.
Replicated Architecture
In a replicated architecture, a copy of the shared application runs locally at each site. Input events
to each application are distributed to all sites and each copy of the shared application is executed
locally at each site. The advantages of this architecture are low network traffic, because only input
events are distributed among the sites, and low response times, since all participants get their
output from local copies of the application. The disadvantages are the requirement of the same
execution environment for the application at each site, and the difficulty in maintaining
consistency.
Conferencing
Conferencing supports collaborative computing and is also called synchronous telecollaboration.
Conferencing is a management service that controls the communication among multiple users via
multiple media, such as video and audio, to achieve simultaneous face-to-face communication.
More precisely, video and audio have the following purposes in a tele-conferencing system:
Video is used in technical discussions to display view-graph and to indicate how many users are
still physically present at a conference. For visual support, workstations, PCs or video walls can
be used.
For conferences with more than three or four participants, the screen resources on a PC or
workstation run out quickly, particularly if other applications, such as shared editors or drawing
spaces, are used. Hence, mechanisms which quickly resize individual images should be used.
Conferencing services control a conference (i.e., a collection of shared state information such as
who is participating in the conference, conference name, start of the conference, policies associated
with the conference, etc,) Conference control includes several functions:
Establishing a conference, where the conference participants agree upon a common state, such
as identity of a chairman (moderator), access rights (floor control) and audio encoding. Conference
systems may perform registration, admission, and negotiation services during the conference
establishment phase, but they must be flexible and allow participants to join and leave individual
media sessions or the whole conference. The flexibility depends on the control model.
Closing a conference.
Adding new users and removing users who leave the conference. Conference states can be stored
(located) either on a central machine (centalised control), where a central application acts as the
repository for all information related to the conference, or in a distributed fashion.
Session Management
Session management is an important part of the multimedia communication architecture. It is the
core part which separates the control, needed during the transport, from the actual transport.
Session management is extensively studied in the collaborative computing area; therefore we
concentrate on architectural and management issues in this area.
Architecture
A session management architecture is built around an entity-session manager which separates the
control from the transport. By creating a reusable session manager, which is separated from the
user-interface, conference-oriented tools avoid a duplication of their effort. The session control
architecture consists of the following components:
Session Manager
Session manager includes local and remote functionalities. Local functionalities may include
(1) Membership control management, such as participant authentication or presentation of
coordinated user interfaces;
(2) Control management for shared workspace, such as floor control
(3) Media control management, such as intercommunication among media agents or
synchronization
(4) Configuration management, such as an exchange of interrelated QoS parameters of selection
of appropriate services according to QoS; and
(5) Conference control management, such as an establishment,
modification and a closing of a conference.
Media agents
Media agents are separate from the session manager and they are not responsible for decisions
specific to each type of media. The modularity allows replacement of agents. Each agent performs
its own control mechanism over the particular medium, such as mute, unmute, change video
quality, start sending, stop sending, etc.
Shared Workspace Agent
The shared workspace agent transmits shared objects (e.g., telepointer coordinate,
graphical or textual object) among the shared application.
Session Control
Each session is described through the session state. This state information is either private or shared
among all session participants. Dependent on the functions, which an application required and a
session control provides, several control mechanisms are embedded in session management:
Floor control: In a shared workspace, the floor control is used to provide access to the shared
workspace. The floor control in shared application is often used to maintain data consistency.
Conference Control: In conferencing applications, Conference control is used. Media control: This
control mainly includes a functionality such as the synchronization of media streams.
Configuration Control: Configuration control includes a control of media quality,
QOS handling, resource availability and other system components to provide a
session according to user’s requirements. Membership control: This may include services, for
example invitation to a session, registration into a session, modification of the membership during
the session etc.