986 PARI VI APPLUCATION LA YER
28.3 MULTIMEDIA IN THE INTERNET
We can divide audio and video services into three broad categories: streaming
audiohideo, streamning live audiohideo, and interactive audiohvideo. Streaming stored
means
a user can listen (or watch) the file after the downloading has started.
28.3.1 Streaming Stored Audio/Video
In the first category, streaming stored audio/video, the files are compressed and stored
on a server. Aclient downloads the files through the Internet. This is sometimes
referred to as on-demand audiolvideo. Examples of stored audio files are songs
symphonies, books on tape, and famous lectures. Examples of stored video files are
movies, TV shows, and music video clips. We can say that streaming stored audiol
video refers to on-demnand requests for compressed audio/video files.
Downloading these types of files from a Web server can be different from down
loading other types of files. To understand the concept, let us discuss three approaches.
cach with a different complexity.
First Approach:Using aWeb Server
TAcompressed audio/video file can be downloaded as a text file. The client (browser) can
use the services of HTTP and send a GET message to download the file. The Web server
cansend the compressed file to the browser. The browser can then use a help application,
normally called amedia player, to play the file) Figure 28.24 shows this approach.
Figure28.24 Using a Web server
Server machine
Client machine
GET: audio/video file
Web
Browser
RESPONSE server
2
Audio/video
file
Media
player
(This approach is very simple and does not involve streaming. However, it has a draw
back. An audio/video file is usually large even after compression. An audio file may co
taintens of megabits, and a video file may contain hundreds of
the file nceds to download completely before it can be megabits,) In this approu
rates, the user nccds some seconds or tens of seconds played. Using contemporay
before the file can be playea.
CHAPTER28 MULTIMEDIA 987
Second Approach: Using aWeb Server with aMetafile
In another approach, the media player is directly connectedto the Web server for down
loading the audio/video file. The Web server stores two files: the actual audio/video hle
and ametafile that holds information about the audio/video file) Figure 28.25 shows
the steps in this approach.
Figure 28.25 Using a Web server with a metafile
Server machine
Client machine
GET: metafile
Browser RESPONSE
2
Web
server
Metafile
GET: audio/video file
Media
player RESPONSE
1. The HTTP client accesses the Web server using the GET message.
2. The information about the metafile comes in the response.
3. The metafile is passed to the mediaplayer.
the audio/video file.
4. The media player uses the URL inthe mnetafile to access
5. The Web server responds.
Third Approach: Using a Media Server
the media player both use
The problem with the second approach is that the browser and
appropriate for retriev
the services of HTTP. HTTP is designed to run over TCP. This is
The reason is that TCP retrans
ing the metafile, but not for retrieving the audio/video file. We
to the philosophy of streaming
mits a lost or damaged segment, which is counter to
need to dismiss TCP and its error control; we need use UDP. However, HTTP. which
for TCP; we need another
accesses the Web server, and the Web server itself are designed
the concept.
server, a media server. Figure 28.26 shows
GET message.
1. TheHTTP client accesses the Web server using a
2. The information about the metafile comes in the
response.
3. The metafile is passed to the media player.
down.
4. The media player uses the URL in the metatile to acccss the media server to
load the file, Downloading can take placc by any protocolthat uses UDP
5. The media server responds,
988 PART VI APPUCATION LAYER
Figure 28.26 Using a media sener
Server ahine
Chent machine
GET: metaile
Web
Brow ser RESPONSE Sener
Metañle
GET: audioDideo file
Media Media
player RESPONSE
Fourth Approach: Using a Media Server and RTSP
The Real-Time Streaming Protocol(RTSP) is a control protocol designed to add
more functionalities to the streaming process. Using RTSP, we can control the playing
of audio/video. RTSP is an out-of-band control protocol that is similar to the second
connectionin FTPFigure 28.27 shows a media server and RTSP.
Figure 28.27 Using a media senver and RTSP
Server machine
Client machine
GET: metafile
Browser Web
RESPONSE Server
2
Metafle
SETUP
RESPONSE
PLAY
Media Media
RESPONSE
player server
Audio/video
Stream
TEARDOWN
RESPONSE
CHAPTER 28 MULTIMEDIA 989
A. The HTTP client accesses the WVch server using a GET
Message.
The infomation about the metafile comesin the
3. The metafile is passcd to tthe media
response.
player.
A. The media player sends aSETUP message tocreate a connection with the media
server.
6. The media server responds.
6. The media player sends a PLAY message to start playing (downloading).
7. The audio/video file is downloaded using another protocol that runs over UDP.
6. The connection is broken using the TEARDOWN message.
6. The media server responds.)
The media player can send othertypes of messages. For example, aPAUSE message tem
porarily stops thedownloading; downloading can be resumed with aPLAY message.
Example: Video on Demand (VOD)
Video On Demand (VOD) allows viewers to select a video from a large number of
available videos and watch it interactively: pause, rewind, fast forward, etc. A viewer
may watch the video in real time or she may download the video into her computer,
portable media player, or to a device such as a digital video recorder (DVR) and watch
it later. Cable TV, satellite TV, and IPTV providers offer both pay-per-view and free
content VOD streaming. Many other companies, such as Amazon video and video
rental companies such as Blockbuster video, also provide [Link] television is an
increasingly popular form of video on demand.
28.3.2 Streaming Live Audio/Video
In the second category, streaming live audio/video, a user listens to broadcast audio and
video through the Internet. Good examples of this type of application are Internet radio
and Internet TV.
There are several similarities between streaming stored audio/video and streaming
live audio/video. They are both sensitive to delay; neither can accept retransmission.
However, there is a difference. In the first application, the communication is unicast and
on-demand. In the second, the communication is multicast and live. Live streaming is
better suited to the multicast services of IP and the use of protocols such as UDP and
RTP (discussed later). However, presently, live streaming is still using TCP and multiple
unicasting instead of multicasting. There is still much progress to be made in this area.
Example: Internet Radio
Internet radio or web radio is awebcast of audio broadcasting service that offers news.
sports, talk, and music via the Internet. It involves a streaming medium that is accessi
ble frorn anywhere in the world. Web radio is offered via the Internet but is similar to
traditional broadcast media: it is noninteractive and cannot be pauscd or replayed like
on-demand services. The largest group of Internet radio providers today includes exist
ing radio stations that simultancously broadcast their output traditionally and over the
Internet. It also includes Internet-only radio stations. In web radio, audiosound is often
compressed by MP3 or similar software and the bits are transported over TCP or UDP
CHAPTER 28 ULIIMEDIA 991
O0:00:01l, the second packet at (00:00:|1, and the third
is a 1-s time diflerence
between what the server sends andpacket at 0000:21. Although there
puter screen, the action is happening in realtime. The what the client sees on thc com
ets is preserved. The I-sdelay is not time relationship between the pack
important. Figure 28.28 showsthe idea.
Figure 28.28 Time relationship
Client Server
Internet
seconds
30 00.00.01 O0.00.0O seconds
30
First Packet
00.00.11 00.00.10
Second Packet
00.00.21 00.00.20
Third Packet
00.00.31 00.00.30
Flow
Arrive and play time Send time
But what happens if the packets arrive with different delays? For examplethe first
packet arrives at 00:00:01 (1-s delay), the second arrives at 00-00: 15 (5-s delay). and
the third arrives at 00:00:27(7-s delay). If the receiver starts playing the first packet
at 00:00:01, it will finish at 00:00:11. However, the next packet has not yet arrived; it
arrives 4 s later. There is a gap between the first and second packets and between the
second and the third as the video is viewed at the remote site. This phenomenon is
called jitter Figure 28.29 shows the situation.
Figure 28.29 Jitter
Server
Client
Internet
00.00.00 seconds
00.00.01 First Packet
00.00.10
0
00.00.15 Second Packet 00.00.20
00.00.27 Third Packe 00.00.30
(00,00.37
Flow
Arrive and play time Send tine
CHAPTER 28 MULTIMEDIA
multicasting. A TCP conncction is only a two-party conncction: we nced multi
party connection for real-time interactive communication.
3. We can use UDP and combine it with another transport facility tocompensate for
the requirements that cannot be provided by UDP. In other words, we use UDP o
provide client-server socket interface., but use another protocol that runs at the top
of the UDP. This is the current choice for multimedia applications. This transport
facility is the Real-time Transport Protocol (RTP), which we discuss next.
28.4.2 RTP
(Real-time TransportProtocol (RTP) is the protocol designed to handle real-time traf
fic on the Internet. RTP does not have a delivery mechanism (multicasting, port nurn
bers, and so on); it must be used with UDP. RTP stands between UDP and the
multimedia application. The literature and standards treat RTP as the transport protocol
(nota transport-layer protocol) that can be thought of as located in the application layer
(see Figure 28.34). The data from multimedia applications are encapsulated in RTP.
which inturn passes them to the transport layer. In other words, the socket interface is
located between RTP and UDP, which implies that we should include the functionality
of RTP in client-server programs that we write for each multimedia application) How
ever, some programming languages provide some facilities to make the programming
task easier. For example, the C language provides an RTP library and the Java language
provides an RTP class for this purpose. If we use the RTP library or the RTP class, we
can think that we have separated the applications from the RTP and the RTP has
become part of the transport layer.
Figure 28.34 RTP location in the TCP/P protocol suite
MPEG MPEG Motion
Application Audio Video JPEG
layer
RTP
Socket Interface
Transport UDP
layer
Network
IP
layer
Data-link
layer Underlying network
Physical technology
layer
J [Link] bit fielkd indicates the ye of the payload Several payload
yÊNNhave tven definenl o far. We list some comnon applications in Table 28 7.
Adiscussion of the tyNs is beyond the scope of this bok.
Table 28,7 Aakal vyvs
Ayicat Arplication
INMu Audio LIN audio G728 audio
1016 ICMA audio Motion JPEG
H.261
G721 audio G722 audio
GS\M audio 10-1| LI6audio 32 MPEGI Video
DVI4audio 14 MPEG audio 33 MPEG2 video
Sequence number. This ield is l6 bits in length. It is used to number the RTP
packets, The sequcnce number of the first packet is chosen randomly; it is incre
mented by I for each subsequent packet. The sequence number is used by the
rceiver to detect lost or out of order packets.
u Timestanp. This is a32-bit field that indicates the time relationship betweenpackets.
The timestamp for the first packet is a random nunber. For each succeeding packet,
the value is the sum of the preceding timestamp plus the time the fist byte is produced
(sampled). The value of the clock tick depends on the application. For example, audio
applications nomally generate chunks of 160bytes; the clock tick for this application
is 160. The timestamp for this application increases 160 for each RTP packet.
O Synchronization source (SSRC) identifier. If there is only one source, this 32-bit
field defines the source. However, if there are several sources, the mixer is the syn
chronization source and the other sources are contributors. The value of the source
identifier is a random number chosen by the source. The protocol provides a strat
egy in case of conflict (two sources start with the same sequence number).
O Contributing source (CSRC) identifier. Each of these 32-bit identifiers (a maxi
the
mum of 15)defines a source. When there is more than one source in a session,
mixer is the synchronization source and the remaining sources are the contributors.
UDP Port
Although RTP is itself a transport-layer protocol, the RTP packet is not encapsulated
directly in an IP datagram. Instead, RTP is treated like an application program and is
encapsulated in a UDP user datagram. However, unlike other application programs, no
well-known port is assigned to RTP. The port can be selected on demand with only
number (an odd
one restriction: The port number must be an even number. The next
number) is used by the companion of RTP, Real-time Transport Control Protocol
(RTCP), which we will discuss in the next section.
RTP uses an even-numbered UDP port.
28.4.3 RTCP
RTP allowsonly one type of message, one that carries data from the source to the destina
tion. Toreally control the session,we need more communication between the participants
in asession. Controlcommunication in this case is assigned to aseparate protocolcalled
1002 PART V APPICATlON LAYER
Real-time Transport Control Protocol (RTCP}We necd to emphasize that the kTCP
payloads are not caried in RIP packets; RTCP is in fact a sister protocol of RTP Th
means that the UDl,as the real transpnt protocol, sometimes caries RTP payloads and
sometimes RTCPpay loads as though they belong toditferent upper-layer protocols,
RCTP packets make an out-of-band control stream that provides two-way feed
back infonnation between the senders and receivers of the multimedia streams(n nar
ticular, RTCP provides the following functions:
c1.(RTCP infoms the sender or senders of multimedia streams about the network per.
formance, which can be directly related to the congestion in the network. Since
multimedia applications use UDP (instead of TCP), there is no way to control the
congestion in the network at the transport layer)This means that, if it is necessary
to control the congestion, it should be done at the application layer. RTCP, as we
will see shortly. gives the clues to the application layer to do so. If the congestion is
observed and reported by the RTCP, an application can use a more aggressive com
pression method toreduce the number of packets and, therefore, to reduce conges
tion, for a trade-off in quality. On the other hand, if no congestion is observed, the
application program can use a less aggressive compression method for a better
quality service.
Information carried in the RTCP packets can be used to synchronize different
streams associated with the same source. A source may use two different sources
tocollect audio or video data, In addition, audio data may be collected from differ
ent microphones and video ata may be collected from different cameras. In gen
eral, two pieces of information are needed to achieve synchronization:
a. Each sender needs an identity. Although each source may have a different
SSRC, RTCP provides one single identity, called a canonical name (CNAME)
for each source. CNAME can be used to correlate different sources and allow
the receiver to combine different sources from the same source. For
example, a
teleconference may have n senders associated with asession, but we may have
m sources (m >n) that contribute to the stream. In this system, we have
only n
CNAMEs, but m SSRCs. A CNAME is in the form of
user@host
in which user is normally the login name of the user and the host is the domain
name of the host.
b. The canonical name cannot per se provide
synchronization. To
sOurces, we need to know the absolute timing of the stream, insynchronize the
addition to the
relative timing provided by the timestamp field in each RTP packet. Thec time
stamp information in each packet gives the relative time relationship of the bits
in the packet to the beginning of the stream: it cannot
relate one streat o
another. The absolute time, the "wall lock" time as it is sometimes refemed
necds to be sent by RTCP packets toenable
3, An RTCP packet can carryextra synchronization.
information about the
lor the receiver, such as the name of he sender (bevondsender that can beuseia
tions for a video. canonical nane) C
CHAPTER 28 MIULTIMEDIA I003
RTCPPackets
Ater discussing the main functions and
Figure 2S.36 shows five common packet purpose
types.
of RTCP, let us discuss its packets.
The
the numeric value of each packet. We needto number next to each box defines
mention
canbe packed as a single payload for UDP because thethat more than one RTCP packet
RTCP packets are smaller than
RIP packets)
Figure 28.36 RlCP packet types
Sender report 200
Receiver report 201
RTCP
packets Source deseription message 202
Bye message 203
Application-specificmessage 204
The format and the exact definition of each field is very involved and beyond the
scope and space of this [Link] briefly discuss the purpose of each packet and relate it
tothe functions previously described.
Sender Report Packet
The sender report packet is sent periodically by the active senders in a session to report
transmission and reception statistics for all RTP packets sent during the interval. The
sender report packet includes the following information:
O The SSRC of the RTP stream.
O The absolute timestamp, which is the combination of the relative timestamp and the
wall clock time, which is the number of seconds elapsed since midnight January 1.
1970. The absolute timestamp. as discussed previously, allows the receiver to syn
chronize different RTP packets.
QThe number of RTP packets and bytes sent from the beginning of the session.
Receiver Report Packet
The receiver report is issued by passive participants, those that do not send RTP pack
[Link] report informs the sender and other receivers about the quality of service. The
feedback information can be used for congestion control at the sender site. A receiver
report includes the following information:
J The SSRCof the RTP stream for which the receiver report has been generated.
O The fraction of packet loss.
J Thelast scquence number.
U The interval jitter.