Real-Time Protocols
RTP/RTCP
Overview
History of streaming media
Streaming performance requirements
Protocol stack for multimedia services
Real-time transport protocol(RTP)
RTP control protocol (RTCP)
Brief history of streaming media
Real-time multimedia streaming
Real-time multimedia applications
Video teleconferencing
Internet Telephony (VoIP)
Internet audio, video streaming
(A-PDUs)
Streaming performance requirements
Sequencing
to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment covers up errors from lost PDU by using
redundancy in most-adjacent-frame
Quality of Service (QoS) feedback from receiver to sender for
operation adjustment
Rate control sender reduces sending rate adaptively to network
congestion
Ideal Timing no jitter
30 seconds
00.00.00
00.00.10
00.00.20
00.00.30
First RTP-PDU
Second RTP-PDU
application
00.00.11
00.00.21
Third RTP-PDU
00.00.31
Send time
Play time
Reality jitter
delay
00.00.00
00.00.10
First RTP-PDU
00.00.20
Second RTP-PDU
00.00.21
00.00.30
00.00.40
Third RTP
-PDU
Fourth
RTP-P
DU
Send time
00.00.11
00.00.25
00.00.35
00.00.37
00.00.41
00.00.47
00.00.51
Play time
Jitter
(contd.)
00.00.00
00.00.10
First RTP-PDU(0)
00.00.20
Second RTP-PDU(1
0)
00.00.21
00.00.30
00.00.40
Third RTP
-PDU(20)
Fourth
RTP-P
DU
Send time
00.00.11
(30)
00.00.25
00.00.35
00.00.37
00.00.41
00.00.47
00.00.18
00.00.28
00.00.38
00.00.48
00.00.51
Play time
00.00.58
Jitter
(contd.)
Playback buffer
At time 00:00:18
At time 00:00:28
At time 00:00:38
Audio silence example:
Not send anything
Why might this cause problems?
sender
Consider audio data
What should the sender do during silence?
Seq no
.1, Tmp
st 1 0 0
Seq no
.2, Tmp
st 2 0 0
Seq no
.3, Tmp
st 3 0 0
silence
Receiver cannot distinguish between loss
and silence
Solution:
Seq no
.4, Tmp
st 6 0 0
Seq no
.5, Tmp
st 700
After receiving no PDUs for a while, next PDU received at
the receiver will reflect a big jump in timestamp, but
have the correct next seq. no. Thus, receiver knows
what happened.
receiver
How does Sequence number and Timestamp help ?
Streaming performance requirements
Sequencing
to report PDU loss
to report PDU reordering
to perform out-of-order decoding
Time stamping and Buffering
for play out
for jitter and delay calculation
Payload type identification
for media interpretation
Error concealment covers up errors from lost PDU by using
redundancy in most-adjacent-frame
Quality of Service (QoS) feedback from receiver to sender for
operation adjustment
Rate control sender reduces sending rate adaptively to network
congestion
Support from transport layers
TCP is not used because:
TCP does retransmissions unbounded delays
No provision for time stamping
TCP does not support multicast
TCP congestion control (slow-start) unsuitable for real-time transport
RTP + UDP usually used for multimedia services
Protocol stack for multimedia services
RTSP
RTP
RTCP
TCP
(till now)
RTP: Introduction
Provides end-to-end transport functions for real-time
applications
Supports different payload types
All RTP and RTCP PDUs are sent to same multicast group
(by all participants)
All RTP PDUs sent to an even-numbered UDP port, 2p
All RTCP PDUs sent to UDP port 2p+1
Transport
layer
Does NOT provide timely delivery or other QoS guarantees
Relies on other protocols like RTCP and lower layers
Application
RTP RTCP
UDP
IP
Data Link
Does NOT assume the underlying network is reliable and
delivers PDUs in sequence
Uses sequence number
Physical
RTP Session
RTP session is sending and receiving of RTP data by a group
of participants
For each participant, a session is a pair of transport
addresses used to communicate with the group
If multiple media types are communicated by the group, the
transmission of each medium constitutes a session.
RTP Synchronization Source
synchronization source - each source of RTP PDUs
Identified by a unique,randomly chosen 32-bit ID (the SSRC)
A host generating multiple streams within a single RTP must
use a different SSRC per stream
RTP Basics of Data Transmission
RTP PDUs
RTP PDU Header
Sampling instant of first data octet
multiple PDUs can have same timestamp
not necessarily monotonic
used to synchronize different
Payload type
media streams
Incremented by one for
each RTP PDU:
PDU loss detection
Restore PDU sequence
Identifies synchronization source
Identifies contributing sources
(used by mixers)
Mixer
RTP mixer - an intermediate system that receives & combines
RTP PDUs of one or more RTP sessions into a new RTP PDU
Stream may be transcoded, special effects may be performed.
A mixer will typically have to define synchronization relationships between
[Link]
Sources that are mixed together become contributing sources (CSRC)
Mixer itself appears as a new source having a new SSRC
Translator
An intermediate system that
Connects two or more networks
Multicasting through a firewall
Modifies stream encoding, changing the streams timing
Transparent to participants
SSRCs remain intact
end system 1
from ES1: SSRC=6
from ES2: SSRC=23
end system 2
transl.1
from ES1: SSRC=6
from ES2: SSRC=23
authorized tunnel
firewall
transl.2
from ES1: SSRC=6
from ES2: SSRC=23
RTP Control Protocol (RTCP)
RTCP specifies report PDUs exchanged between sources and
destinations of multimedia information
receiver reception report
sender report
source description report
Reports contain statistics such as the number of RTP-PDUs sent,
number of RTP-PDUs lost, inter-arrival jitter
Used by application to modify sender transmission rates and for
diagnostics purposes
RTCP message types
Typically, several RTCP PDUs of different types are transmitted
in a single UDP PDU
Sender/Receiver report PDUs
V
RC
PT=200/201 SR/RR
Length (16 bits)
SSRC of Sender
Header
NTP Timestamp, most significant word
NTP Timestamp, least significant word
RTP Timestamp
Senders PDU Count
Sender
Info
Senders Octet Count
SSRC_1 (SSRC of the 1st Source)
Fraction Lost
Cumulative Number of PDU Lost
Extended Highest sequence Number Received
Interarrival Jitter
Report
Block 1
Last SR (LSR)
Delay Since Last SR (DLSR)
SSRC_2 (SSRC of the 2nd Source)
Profile-Specific Extensions
Report
Block 2
Ethereal capture for RTP-PDU
Basic header
Ethereal capture for RTCP-PDU
header of SR report
sender info
receiver report block
SDES items
Synchronization of streams using RTCP
RTP audio
RTCP audio
RTP video
RTP video
Internetwork
Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks
timestamps are not tied to the wall-clock time, or each other!
Each RTCP sender-report PDU contains (for most recently generated PDU in
associated RTP stream):
The timestamp of RTP PDU
The wall-clock time for when PDU was created
Receivers can use this association to synchronize the playout of audio and video
RTCP bandwidth scaling
Problem
What happens when there is
one sender and many receivers?
RTCP reports scale linearly with
the number of participants and
would match or exceed the
amount of RTP data! More
overhead than useful data!
Solution
RTCP attempts to limit its
traffic to 5% of the session
bandwidth to ensure it can
scale!
RTCP gives 75% of this rate
to the receivers; and the
remaining 25% to the
sender.
Example
Suppose one sender, sending
video at a rate of 2 Mbps. Then
RTCP attempts to limit its traffic
to 100 Kbps.
The 75 kbps is equally shared
among receivers:
With R receivers, each
receiver gets to send RTCP
traffic at 75/R kbps.
Sender gets to send RTCP traffic
at 25 kbps.
Real-Time Streaming Protocol (RTSP)
Application layer protocol (default port 554)
Usually runs on RTP for stream & TCP for control
Provides the control channel
Uses out-of-band signaling
Usable for Live broadcasts / multicast
Also known as Network remote control for multi-media
servers.
RTSP Overview
Web Server
web
browser
HTTP
presentation descriptor
Presentation
descriptor
media
player
Web Server/Media server
RTSP
pres. desc,streaming commands
RTP/RTCP
audio/video content
RTSP Methods
OPTIONS
CS
CS
determine capabilities of server/client
DESCRIBE
CS
get description of media stream
ANNOUNCE
CS
announce new session description
SETUP
CS
create media session
RECORD
CS
start media recording
PLAY
CS
start media delivery
PAUSE
CS
pause media delivery
REDIRECT
CS
redirection to another server
TEARDOWN
CS
immediate teardown
SET_PARAMETER
CS
change server/client parameter
GET_PARAMETER
CS
read server/client parameter
RTSP Session
Default port
554
RTSP SETUP
RTSP OK
RTSP
server
RTSP PLAY
RTSP OK
RTSP TEARDOWN
RTSP OK
TCP
RTSP
client
get UDP port
data
source
RTP VIDEO
RTP AUDIO
RTCP
media server
choose
UDP port
UDP
AV
subsyste
m
media player
Example:Media on demand (Unicast)
Media server A
[Link]
Media server V
[Link]
Client C
Web server W
-holds the media descriptors
RTSP Message sequence
C -> W : GET/[Link] HTTP/1.1
Host: [Link]
Accept: application/sdp
W-> C : HTTP/1.0 200 OK
Content-Type: application/sdp
C-> A : SETUP rtsp://[Link]/twister/[Link] RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 12345678
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
server_port=5000-5001
C->V : SETUP rtsp://[Link]/twister/[Link] RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 23456789
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
server_port=5002-5003
W
V
C
RTSP Message sequence
(contd.)
C->V: PLAY rtsp://[Link]/twister/video RTSP/1.0
Cseq: 2
Session: 23456789
W
V->C: RTSP/1.0 200 OK
Cseq: 2
Session: 23456789
RTP-Info: url=rtsp://[Link]/twister/video;
seq=12312232;
C->A: PLAY rtsp://[Link]/twister/[Link] RTSP/1.0
Cseq: 2
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 2
Session: 12345678
RTP-Info: url=rtsp://[Link]/twister/[Link];
seq=876655;
V
C
RTSP Message sequence
(contd.)
C->A: TEARDOWN rtsp://[Link]/twister/[Link] RTSP/1.0
Cseq: 3
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 3
V
C->V: TEARDOWN rtsp://[Link]/twister/video RTSP/1.0
Cseq: 3
Session: 23456789
V->C: RTSP/1.0 200 OK
Cseq: 3
References
[1] B. A. Forouzan, TCP/IP Protocol Suite,
Third edition,
[2] H. Schulzrinne, S. Casner, R. Frederick and V.
Jacobson, "RTP: a transport protocol for real-time
applications", RFC 3550, July 2003.
[3] H. Schulzrinne, A. Rao and R. Lanphier, "Real Time
Streaming Protocol (RTSP)", RFC 2326, April 1998.
RTCP compound PDU
receiver
report
source 3
SDES
compound PDU
(single UDP datagram)
SSRC
receiver
report
source 2
RTCP PDU 2
SSRC
sender
report
SSRC
SR
SSRC
RTCP PDU 1
CNAME PHONE
Example
source 1 reports, there are 2 other sources
receiver
report
source 2
receiver
report
source 3
SSRC
sender
report
SSRC
SR
SSRC
RTCP PDU
RTCP processing in Translators
SR sender information : Does not generate their own sender
information(most of the times), but forwards the SR PDUs
received from one side to other
RR reception report blocks : Does not generate their own
RR reports (most of the times), but forwards RR reports
received from one side to another. SSRC are left intact
SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A translator about to
cease forwarding, send a BYE PDU to each connected nodes
RTCP processing in Mixers
SR sender information : Generates its own SR info. Because
the characteristics of source stream is lost in the mix. The SR
info is sent in same direction as the mixed stream
RR reception report blocks : Generates its own reports for
sources in each cloud and sends them only to same cloud
SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
BYE : Forwards BYE PDU unchanged. A mixer about to
cease forwarding, send a BYE PDU to each connected nodes
Source description PDUs
May contain:
a CNAME item (canonical identifier/name)
a NAME item (real user name)
an EMAIL item
a PHONE item
a LOC item (geographic location)
a TOOL item (application name)
a NOTE item (transient msg, e.g. for status)
a PRIV item (private extension)
CNAME=1
length
user and domain name
Value
1
2
3
4
5
6
7
8