Module - IV

Application Layer

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Applications Layer – allows user to interface with the network!

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Application Layer – Provides the interface between the
applications on either end of the network.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How data requests occur & are
filled
• Client/server model
Advantages:
Centralized administration
Security is easier to enforce
• Application layer services and protocols
• Peer-to-peer networking and applications

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

Figure 25-2

Client-Server Model

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Client/Server Model
• Client – device requesting information (initiates the data exchange)
Can also UPLOAD data to the servers
• Server – device responding to the request
Relies on support from the lower layer functions to distinguish
between services and conversations.
Server relies on a service called a server daemon – runs in the
background and ‘listens’ for requests for that service. It can then
exchange messages as appropriate & send requested data.
• Examples:
E-mail Client on an employee computer issues a request to the e-mail
server for any unread e-mail. The server responds by sending the e-
mail to the client.
Conversations can originate with either party.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Peer-to-Peer (P2P) Network Model
• Two or more computers are connected and are
able to share resources without having a
dedicated server
• Every end device can function as a client or
server on a ‘per request’ basis
• Resources are decentralized (information can
be located anywhere)
• Difficult to enforce security and policies
• User accounts and access rights have to be set
individually on each peer device
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 P2P Applications
• Running applications in hybrid mode allows for a
centralized directory of files even though the files
themselves may be on multiple machines
• Unlike P2P networks, a device can act as both the
client and server within the same communication
• Each device must provide a user interface and run
a background service.
• Can be used on P2P networks, client/server
networks and across the internet.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 P2P Applications Example

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Common Port Numbers
• TCP
FTP – 20-21
Telnet – 23
SMTP – 25
DNS – 53 (Both TCP & UDP)
HTTP – 80

• UDP
DHCP – 67 & 68
POP – 110

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 The TELNET Protocol

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 TELNET vs. telnet

• TELNET is a protocol that provides “a

general, bi-directional, eight-bit byte oriented
communications facility”.
• telnet is a program that supports the
TELNET protocol over TCP.
• Many application protocols are built upon the
TELNET protocol.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 The TELNET Protocol

• TCP connection
• data and control over the same connection.
• Network Virtual Terminal
• negotiated options

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Network Virtual Terminal

• intermediate representation of a generic

terminal.
• provides a standard language for
communication of terminal control functions.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Playing with TELNET
• You can use the telnet program to play with the
TELNET protocol.
• telnet is a generic TCP client.
– Sends whatever you type to the TCP socket.
– Prints whatever comes back through the TCP socket
– Useful for testing TCP servers (ASCII based protocols).
• Many Unix systems have these servers running (by
default):
– echo port 7 discard port 9
– daytime port 13 chargen port 19

McGraw-Hill TELNET 15
©The McGraw-Hill Companies, Inc., 2001
 telnet hostname port
> telnet [Link] 7
Trying [Link]...
Connected to [Link]
([Link]).
Escape character is '^]'.
Hi mehmet
Hi mehmet
stop it
stop it
^]
telnet> quit
Connection closed.
McGraw-Hill TELNET 16
©The McGraw-Hill Companies, Inc., 2001
 The Domain Name System

17
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 A primary server loads all information
from the disk file; the secondary server
loads all information from the primary
server.

18
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS In The Internet

Generic Domain

Country Domain

Inverse Domain

19
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS in the Internet

20
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Generic domains

21
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Generic domain labels

Label Description

com Commercial organizations

edu Educational institutions

gov Government institutions

int International organizations

mil Military groups

net Network support centers

org Nonprofit organizations

22
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Table 25.2 New generic domain labels

Label Description

aero Airlines and aerospace companies

biz Businesses or firms (similar to com)

coop Cooperative business organizations

info Information service providers

museum Museums and other nonprofit organizations

name Personal names (individuals)

pro Professional individual organizations

23
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Country domains

24
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Inverse domain

25
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Hostnames

• IP Addresses are great for computers

– IP address includes information used for routing.
• IP addresses are tough for humans to
remember.
• IP addresses are impossible to guess.
– ever guessed at the name of a WWW site?

26
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 The Domain Name System

• The domain name system is usually used to

translate a host name into an IP address .

• Domain names comprise a hierarchy so that

names are unique, yet easy to remember.

27
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Hierarchy

edu com org jp

rpi albany

28
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Host name structure
• Each host name is made up of a sequence of
labels separated by periods.
– Each label can be up to 63 characters
– The total name can be at most 255 characters.
• Examples:
– [Link]
– [Link]
– [Link]

29
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Domain Name

• The domain name for a host is the sequence of

labels that lead from the host (leaf node in the
naming tree) to the top of the worldwide
naming tree.

• A domain is a subtree of the worldwide

naming tree.

30
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Top level domains
• edu, gov, com, net, org, mil, …
• Countries each have a top level domain (2
letter domain name).
• New top level domains include:
.aero .biz .coop .info .name .pro

31
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Organization
• Distributed Database
– The organization that owns a domain name is
responsible for running a DNS server that can
provide the mapping between hostnames within
the domain to IP addresses.
– So - some machine run by RPI is responsible
for everything within the [Link] domain.

32
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Distributed Database
• There is one primary server for a domain,
and typically a number of secondary servers
containing replicated databases.
[Link] DNS server

[Link] [Link]
[Link]
[Link]
DB
DNS DB DNS
DNSDBDB

Authoritative Replicas

33
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Clients
• A DNS client is called a resolver.

• A call to gethostbyname()is handled by a

resolver (typically part of the client).

• Most Unix workstations have the file

/etc/[Link] that contains the
local domain and the addresses of DNS
servers for that domain.
34
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 /etc/[Link]

domain [Link]
[Link]
[Link]

35
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 nslookup

• nslookup is an interactive resolver that

allows the user to communicate directly with a
DNS server.

• nslookup is usually available on Unix

workstations. (dig and host are also DNS
clients).

36
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Servers

• Servers handle requests for their domain

directly.
• Servers handle requests for other domains by
contacting remote DNS server(s).
• Servers cache external mappings.

37
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Server - Server Communication
• If a server is asked to provide the mapping
for a host outside it’s domain (and the
mapping is not in the server cache):
– The server finds a nameserver for the target
domain.
– The server asks the nameserver to provide the
host name to IP translation.
• To find the right nameserver, use DNS!

38
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Data

• DNS databases contain more than just

hostname-to-address records:
– Name server records NS
– Hostname aliases CNAME
– Mail Exchangers MX
– Host Information HINFO

39
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 The Root DNS Server

• The root server needs to know the address of

1st (and many 2nd) level domain nameservers.

edu com org jp

rpi albany
40
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Server Operation

• If a server has no clue about where to find the

address for a hostname, ask the root server.
• The root server will tell you what nameserver
to contact.
• A request may get forwarded a few times.

41
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Message Format

HEADER
HEADER
QUERIES
QUERIES
Response
Response RESOURCE
RESOURCE RECORDS
RECORDS
Response
Response AUTHORITY
AUTHORITY RECORDS
RECORDS
Response
Response ADDITIONAL
ADDITIONAL INFORMATION
INFORMATION

42
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Message Header

• query identifier
16 bit fields

• flags
• # of questions

}
• # of RRs
• # of authority RRs Response
• # of additional RRs

43
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Message Flags

• QR: Query=0, Response=1

• AA: Authoritative Answer
• TC: response truncated (> 512 bytes)
• RD: recursion desired
• RA: recursion available
• rcode: return code

44
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Recursion

• A request can indicate that recursion is desired

- this tells the server to find out the answer
(possibly by contacting other servers).
• If recursion is not requested - the response
may be a list of other name servers to contact.

45
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Question Format

• Name: domain name (or IP address)

• Query type (A, NS, MX, …)

• Query class (1 for IP)

46
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Response Resource Record

• Domain Name
• Response type
• Class (IP)
• Time to live (in seconds)
• Length of resource data
• Resource data

47
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Resource Records
• Each name server maintains a collection of resource records
(Name, Value, Type, Class, TTL)
– Each record is a translation based on type
– Name/Value: not necessarily host names to IP addresses
• Type (some examples)
– A: Name = full domain name, Value = IP address
– NS: Value gives domain name for host running name server that
knows how to resolve names within specified domain.
– CNAME: Value gives canonical name for particle host; used to
define aliases.
– MX: Value gives domain name for host running mail server that
accepts messages for specified domain.
• Class: allow other entities (other than NIC) to define types
– IN is what is used by the Internet
• TTL: how long the resource record is valid
48
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 DNS Resource Record
The resource record will be in this format
Domain_name time_to_live class type value
Domain_name - it tells the domain name
time_to_live - the existance of record for 1day/hr/min
Class – it is generally IN(internet information)
Type – it denotes a Host/name server/mail server etc..
Value – it can be a number/domain name/Ascii string

Eg:
[Link] 86400 IN NS [Link]
[Link] 86400 IN MX [Link]

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 UDP & TCP

• Both UDP and TCP are used:

– TCP for transfers of entire database to secondary
servers (replication).
– UDP for lookups
– If more than 512 bytes in response - requestor
resubmits request using TCP.

50
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Name to Address Conversion

• There is a library of functions that act as DNS

client (resolver).
– you don’t need to write DNS client code to use
DNS!
• With some OSs you need to explicitly link
with the DNS resolver library:
-lnsl (nsl is “Name Server Library”)

51
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 FTP - File Transfer Protocol
TFTP – Trivial FTP

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Overview
File Transfer Protocol (RFC 959)
• Why FTP?
• FTP’s connections
• FTP in action
• FTP commands/responses
Trivial File Transfer Protocol (RFC 1350)
• TFTP and TFTP’s message formats
• FTP and TFTP compared

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Why do we need a FTP
Service?
Purpose: To Transfer files between two
computers
Goals of FTP Service
• Promote sharing of files (programs and/or data)
• Encourage indirect/implicit use of remote
computers
• Shield users from variations in file storage among
hosts
• Transfer data reliably and efficiently
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Problems of File Transfer

• At first, file transfer may seem simple

• Heterogeneous systems use different:
– Operating Systems
– Character Sets
– Naming Conventions
– Directory Structures
– File Structures and Formats
• FTP need to address and resolve these problems

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 FTP’s2 Connections
client * Persistent
*Non-persistent
Insulates
Server is
command
users
listening from
data
on port “raw”
21
* Routes
and “raw”
reply
*Server FTP commands
connection
uses port 20
FTP
for commands
connection
connection requests
* Receives
for data server’s replies
connections
User
Interface
server

User Control Server

Protocol Protocol
Interpreter Connection 21Interpreter

User Data Server

Data Transfer Data Transfer
Function Connection 20Function

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 FTP’s Connections – Establishment
2 ftp> open [Link]
Connected to [Link]
220 strauss FTP server ready.
client
530 Please login with USER and PAS
331 Password req for haggerty.
User Password:
230 User haggerty logged in.
Interface ftp>
USER
PASS haggerty
mypass
server

User Server
Protocol Control Protocol
Interpreter Connection Interpreter

User Data Server

Data Transfer Data Transfer
Function Connection Function

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 FTP’s2 Connections
Data Transfer
client ls [Link]
-rw-r--r-- haggerty [Link]
User
Interface
226
200
PORT
150 Closing
LIST
Port Data Connection
192,168,100,173,19,137
[Link]
Data Command
ConnectionSucessful
will be open shortly
server

User Server
Protocol Control Protocol
Interpreter Connection Interpreter
Passive open on
Port 5001
User Data Server
Data Transfer Data Transfer
Function Connection Function

McGraw-Hill
Establish Data ©The McGraw-Hill Companies, Inc., 2001
 FTP’s 2
Connections – Connection Closing

bye
client

User
Interface
221 Service Closing
QUIT
server

User Server
Protocol Control Protocol
Interpreter Connection Interpreter

User Data Server

Data Transfer Data Transfer
Function Connection Function

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 FTP Connections

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 FTP Connection
Client Server
ftp> open server SYN
SYN|ACK
ACK
220 Service Ready
ACK
ftp> USER
haggerty ACK
331 User
ACK OK,password?
tp> PASS mypass
ACK
230 User login OK
ACK
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 FTP Data Transfer

Client Server
PORT 192,168,0,173,19,137

ACK 200 Command

NLST [Link] ACK Successful

SYN
SYN-
AACK
CK
150 Data
ACK Connection will be
NAME LIST
open shortly
FIN
Control connection FIN-ACK
Data Connection ACK
226 Closing Data
ACK
Connection
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 FTP Client Commands (issued by user interface)

Command Description
get filename Retrieve file from server

mget filename* Retrieve multiple files from server*

put filename Copy local file to server

mput filename* Copy multiple local files to server*

open server Begin login to server

bye / close / exit Logoff server

ls / dir List files in current remote dir on server

lcd Change local directory

cd Change remote directory

rhelp / remotehelp Lists commands the server accepts

* Sent to server as multiple command by User

McGraw-Hill
Protocol Interpreter
©The McGraw-Hill Companies, Inc., 2001
 Example FTP Responses

• 120 Service will be ready shortly

• 200 Command OK
• 230 User login OK
• 331 User name OK; password is needed
• 421 Service not available
• 530 User not logged in
• 552 Requested action aborted; exceeded storage
allocation

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Summary of FTP connections
• FTP has 2 connections
- Control (persistent connection)
- Server issues a passive open on well-known 21
- Client uses an ephemeral port to issue active open
- Server ultimately closes control connection
- Data (ephemeral connection)
- Client issues passive open on an ephemeral port
- Client sends this port to server via PORT command
- Server receives the port number and issues active open
using its well-known 20 to the received ephemeral port

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Note:

TFTP uses the services of UDP on the

well-known port 69.

TCP/IP Protocol 66
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Trivial FTP (TFTP)
• Used only to read and write files
from/to a remote server
– Cannot list directories
• Useful for bootstrapping diskless
systems
– Workstations
– X terminals
• Simple and small:
– 5 message formats
– Runs on UDP
– Designed to fit in ROM
– Uses a “stop and wait” protocol
– NO BUILT IN SECURITY FEATURES (login)
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 FTP vs. TFTP
• FTP provides (minimal) security through
login procedure
• TFTP has NO login procedure
• FTP Provides a reliable service through
its use of TCP
• TFTP must handle its own
retransmissions since it uses UDP
• FTP uses two connections
• TFTP uses one connection (stop and wait)
• FTP provides many commands
• TFTP can only read and write files

McGraw-Hill
Slide from William Boyer ©The McGraw-Hill Companies, Inc., 2001
 Hyper Text Transfer Protocol

69
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 HTTP uses the services of
TCP on well-known port 80.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

Figure 24-1

HTTP transaction

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

Figure 24-5

URL

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Contents
• Main ingredients of the Web
– URL, HTML, and HTTP
• Key properties of HTTP
– Request-response, stateless, and resource meta-data
• Web components
– Clients, proxies, and servers
– Caching vs. replication
• Interaction with underlying network protocols
– DNS and TCP
– TCP performance for short transfers
– Parallel connections, persistent connections, pipelining

McGraw-Hill
73
©The McGraw-Hill Companies, Inc., 2001
 Web History
• Before the 1970s-1980s
– Internet used mainly by researchers and academics
– Log in remote machines, transfer files, exchange e-mail
• Late 1980s and early 1990s
– Initial proposal for the Web by Berners-Lee in 1989
– Competing systems for searching/accessing documents
• Gopher, Archie, WAIS (Wide Area Information Servers), …
• All eventually subsumed by the World Wide Web
• Growth of the Web in the 1990s
– 1991: first Web browser and server
– 1993: first version of Mosaic browser

McGraw-Hill
74
©The McGraw-Hill Companies, Inc., 2001
 Enablers for Success of the Web
• Internet growth and commercialization
– 1988: ARPANET gradually replaced by the NSFNET
– Early 1990s: NSFNET begins to allow commercial traffic
• Personal computer
– 1980s: Home computers with graphical user interfaces
– 1990s: Power of PCs increases, and cost decreases
• Hypertext
– 1945: Vannevar Bush’s “As We May Think”
– 1960s: Hypertext proposed, and the mouse invented
– 1980s: Proposals for global hypertext publishing systems

McGraw-Hill
75
©The McGraw-Hill Companies, Inc., 2001
 Main Components: URL
• Uniform Resource Identifier (URI)
– Denotes a resource independent of its location or value
– A pointer to a “black box” that accepts request methods
• Formatted string
– Protocol for communicating with server (e.g., http)
– Name of the server (e.g., [Link])
– Name of the resource (e.g., [Link])
• Name (URN), Locator (URL), and Identifier (URI)
– URN: globally unique name, like an ISBN # for a book
– URI: identifier representing the contents of the book
– URL: location of the book

McGraw-Hill
76
©The McGraw-Hill Companies, Inc., 2001
 Main Components: HTML
• HyperText Markup Language (HTML)
– Representation of hyptertext documents in ASCII format
– Format text, reference images, embed hyperlinks
– Interpreted by Web browsers when rendering a page
• Straight-forward and easy to learn
– Simplest HTML document is a plain text file
• Easy to add formatting, references, bullets, etc.
– Automatically generated by authoring programs
• Tools to aid users in creating HTML files
• Web page
– Base HTML file referenced objects (e.g., images)
– Each object has its own URL

McGraw-Hill
77
©The McGraw-Hill Companies, Inc., 2001
 Main Components: HTTP

• HyperText Transfer Protocol (HTTP)

– Client-server protocol for transferring resources
– Client sends request and server sends response
• Important properties of HTTP
– Request-response protocol
– Reliance on a global URI
– Resource metadata
– Statelessness telnet [Link] 80

– ASCII format GET /~jrex/ HTTP/1.1

Host: [Link] 78
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Example: HyperText Transfer Protocol

GET /courses/archive/spring06/cos461/
HTTP/1.1
Host: [Link] Request
User-Agent: Mozilla/4.03
<CRLF>

HTTP/1.1 200 OK
Date: Mon, 6 Feb 2006 13:09:03 GMT
Server: Netscape-Enterprise/3.5.1
Last-Modified: Mon, 6 Feb 2006 11:12:23 GMT
Response
Content-Length: 21
<CRLF>
Site under construction
McGraw-Hill
79
©The McGraw-Hill Companies, Inc., 2001
 HTTP: Request-Response Protocol
• Client program • Server program
– Running on end host – Running on end host
– Requests service – Provides service
– E.g., Web browser – E.g., Web server
GET /[Link]

McGraw-Hill
“Site under construction” 80
©The McGraw-Hill Companies, Inc., 2001
 HTTP Request Message
• Request message sent by a client
– Request line: method, resource, and protocol version
– Request headers: provide information or modify request
– Body: optional data (e.g., to “POST” data to the server)
request line
(GET, POST, GET /somedir/[Link] HTTP/1.1
HEAD commands) Host: [Link]
headerUser-agent: Mozilla/4.0
Connection: close
lines
Accept-language:fr
Carriage return,
line feed (extra carriage return, line feed)
indicates end
of message
McGraw-Hill
81
©The McGraw-Hill Companies, Inc., 2001
 Example: Conditional GET
Request
• Fetch resource only if it has changed at the server
GET /courses/archive/spring06/cos461/ HTTP/1.1
Host: [Link]
User-Agent: Mozilla/4.03
If-Modified-Since: Mon, 6 Feb 2006 11:12:23 GMT
<CRLF>
• Server avoids wasting resources to send again
– Server inspects the “last modified” time of the resource
– … and compares to the “if-modified-since” time
– Returns “304 Not Modified” if resource has not changed
– …. or a “200 OK” with the latest version otherwise

McGraw-Hill
82
©The McGraw-Hill Companies, Inc., 2001
 HTTP Response Message
• Response message sent by a server
– Status line: protocol version, status code, status phrase
– Response headers: provide information
– Body: optional data
status line
(protocol
HTTP/1.1 200 OK
status code Connection close
status phrase) Date: Thu, 06 Aug 1998 12:00:15 GMT
header
Server: Apache/1.3.0 (Unix)
lines Last-Modified: Mon, 22 Jun 1998 …...
Content-Length: 6821
Content-Type: text/html
data, e.g.,
requested data data data data data ...
83
HTML file
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Request Methods and Response Codes
• Request methods include
– GET: return current value of resource, run program, …
– HEAD: return the meta-data associated with a resource
– POST: update a resource, provide input to a program, …
• Response code classes
– 1xx: informational (e.g., “100 Continue”)
– 2xx: success (e.g., “200 OK”)
– 3xx: redirection (e.g., “304 Not Modified”)
– 4xx: client error (e.g., “404 Not Found”)
– 5xx: server error (e.g., “503 Service Unavailable”)
• Note similarities to File Transfer Protocol (FTP)

McGraw-Hill
84
©The McGraw-Hill Companies, Inc., 2001
 HTTP Resource Meta-Data
• Meta-data
– Information relating to a resource
– … but not part of the resource itself
• Example meta-data
– Size of a resource
– Type of the content
– Last modification time
• Concept borrowed from e-mail protocols
– Multipurpose Internet Mail Extensions (MIME)
– Data format classification (e.g., Content-Type: text/html)
– Enables browsers to automatically launch a viewer

McGraw-Hill
85
©The McGraw-Hill Companies, Inc., 2001
 Stateless Protocol
• Stateless protocol
– Each request-response exchange treated independently
– Clients and servers not required to retain state
• Statelessness to improve scalability
– Avoid need for the server to retain info across requests
– Enable the server to handle a higher rate of requests
• However, some applications need state
– To uniquely identify the user or store temporary info
– E.g., personalize a Web page, compute profiles or access statistics by
user, keep a shopping cart, etc.
– Lead to the introduction of “cookies” in the mid 1990s

McGraw-Hill
86
©The McGraw-Hill Companies, Inc., 2001
 Cookies
• Cookie
– Small state stored by client on behalf of server
– Included in future requests to the server
Request

Response
Set-Cookie: XYZ

Request
Cookie: XYZ

McGraw-Hill
87
©The McGraw-Hill Companies, Inc., 2001
 Cookies Examples
client server en
usual http request da try i
Cookie file
server tab n b
msg as ac
usual http response creates ID e ke
ebay: 8734 + nd
Set-cookie: 1678 for user
Cookie file 1678http request
usual
amazon: 1678 msg cookie- e s s
ebay: 8734 ac c
cookie:
usual 1678
http response
specific
action

ss
one week later: msg

ce
ac
Cookie file usual http request
msg cookie-
amazon: 1678
ebay: 8734 cookie:
usual 1678
http response
spectific
msg action

McGraw-Hill
88
©The McGraw-Hill Companies, Inc., 2001
 Web Components
• Clients
– Send requests and receive responses
– Browsers, spiders, and agents
• Servers
– Receive requests and send responses
– Store or generate the responses
• Proxies
– Act as a server for the client, and a client to the server
– Perform extra functions such as anonymization, logging,
transcoding, blocking of access, caching, etc.

McGraw-Hill
89
©The McGraw-Hill Companies, Inc., 2001
 Web Browser
• Generating HTTP requests
– User types URL, clicks a hyperlink, or selects bookmark
– User clicks “reload”, or “submit” on a Web page
– Automatic downloading of embedded images
• Layout of response
– Parsing HTML and rendering the Web page
– Invoking helper applications (e.g., Acrobat, PowerPoint)
• Maintaining a cache
– Storing recently-viewed objects
– Checking that cached objects are fresh

McGraw-Hill
90
©The McGraw-Hill Companies, Inc., 2001
 Typical Web Transaction
• User clicks on a hyperlink
– [Link]
• Browser learns the IP address of the server
– Invokes gethostbyname([Link])
– And gets a return value of [Link]
• Browser establishes a TCP connection
– Selects an ephemeral port for its end of the connection
– Contacts [Link] on port 80
• Browser sends the HTTP request
– “GET /[Link] HTTP/1.1
Host: [Link]”

McGraw-Hill
91
©The McGraw-Hill Companies, Inc., 2001
 Typical Web Transaction (Continued)

• Browser parses the HTTP response message

– Extract the URL for each embedded image
– Create new TCP connections and send new requests
– Render the Web page, including the images

• Opportunities for caching in the browser

– HTML file
– Each embedded image
– IP address of the Web site

McGraw-Hill
92
©The McGraw-Hill Companies, Inc., 2001
 Web Server
• Web site vs. Web server
– Web site: collections of Web pages associated with a
particular host name
– Web server: program that satisfies client requests for Web
resources
• Handling a client request
– Accept the TCP connection
– Read and parse the HTTP request message
– Translate the URL to a filename
– Determine whether the request is authorized
– Generate and transmit the response

McGraw-Hill
93
©The McGraw-Hill Companies, Inc., 2001
 Web Server: Generating a Response
• Returning a file
– URL corresponds to a file (e.g., /www/[Link])
– … and the server returns the file as the response
– … along with the HTTP response header
• Returning meta-data with no body
– Example: client requests object “if-modified-since”
– Server checks if the object has been modified
– … and simply returns a “HTTP/1.1 304 Not Modified”
• Dynamically-generated responses
– URL corresponds to a program the server needs to run
– Server runs the program and sends the output to client

McGraw-Hill
94
©The McGraw-Hill Companies, Inc., 2001
 Hosting: Multiple Sites Per
Machine
• Multiple Web sites on a single machine
– Hosting company runs the Web server on behalf of multiple sites
(e.g., [Link] and [Link])
• Problem: returning the correct content
– [Link]/[Link] vs. [Link]/[Link]
– How to differentiate when both are on same machine?
• Solution #1: multiple servers on the same machine
– Run multiple Web servers on the machine
– Have a separate IP address for each server
• Solution #2: include site name in the HTTP request
– Run a single Web server with a single IP address
– … and include “Host” header (e.g., “Host: [Link]”)

McGraw-Hill
95
©The McGraw-Hill Companies, Inc., 2001
 Hosting: Multiple Machines Per
Site
• Replicating a popular Web site
– Running on multiple machines to handle the load
– … and to place content closer to the clients
• Problem: directing client to a particular replica
– To balance load across the server replicas
– To pair clients with nearby servers
• Solution #1: manual selection by clients
– Each replica has its own site name
– A Web page lists the replicas (e.g., by name, location)
– … and asks clients to click on a hyperlink to pick

McGraw-Hill
96
©The McGraw-Hill Companies, Inc., 2001
 Hosting: Multiple Machines Per
Site
• Solution #2: single IP address, multiple machines
– Same name and IP address for all of the replicas
– Run multiple machines behind a single IP address

Load Balancer
[Link]

– Ensure all packets from a single

TCP connection go to the same replica

McGraw-Hill
97
©The McGraw-Hill Companies, Inc., 2001
 Hosting: Multiple Machines Per Site
• Solution #3: multiple addresses, multiple machines
– Same name but different addresses for all of the replicas
– Configure DNS server to return different addresses

[Link] [Link]
Internet

[Link]
McGraw-Hill
98
©The McGraw-Hill Companies, Inc., 2001
 Caching vs. Replication
• Motivations for moving content close to users
– Reduce latency for the user
– Reduce load on the network and the server
– Reduce cost for transferring data on the network
• Caching
– Replicating the content “on demand” after a request
– Storing the response message locally for future use
– May need to verify if the response has changed
– … and some responses are not cacheable
• Replication
– Planned replication of the content in multiple locations
– Updating of resources is handled outside of HTTP
– Can replicate scripts that create dynamic responses

McGraw-Hill
99
©The McGraw-Hill Companies, Inc., 2001
 Caching vs. Replication
(Continued)
• Caching initially viewed as very important in HTTP
– Many additions to HTTP to support caching
– … and, in particular, cache validation
• Deployment of caching proxies in the 1990s
– Service providers and enterprises deployed proxies
– … to cache content across a community of users
– Though, sometimes the gains weren’t very dramatic
• Then, content distribution networks emerged
– Companies (like Akamai) that replicate Web sites
– Host all (or part) of a Web site for a content provider
– Place replicas all over the world on many machines

McGraw-Hill
100
©The McGraw-Hill Companies, Inc., 2001
 TCP Interaction: Multiple
Transfers
• Most Web pages have multiple objects
– E.g., HTML file and multiple embedded images
• Serializing the transfers is not efficient
– Sending the images one at a time introduces delay
– Cannot start retrieving second images until first arrives
• Parallel connections
– Browser opens multiple TCP connections (e.g., 4)
– … and retrieves a single image on each connection
• Performance trade-offs
– Multiple downloads sharing the same network links
– Unfairness to other traffic traversing the links

McGraw-Hill
101
©The McGraw-Hill Companies, Inc., 2001
 TCP Interaction: Short Transfers
• Most HTTP transfers are
short
– Very small request message
(e.g., a few hundred bytes)
initiate TCP
connection
– Small response message RTT
request
(e.g., a few kilobytes) file
time to
• TCP overhead may be big RTT
transmit
file
– Three-way handshake to file
received
establish connection
time
– Four-way handshake to tear time

down the connection 102

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 TCP Interaction: Short Transfers
• Round-trip time estimation
– Very large at the start of a connection (e.g., 3 seconds)
– Leads to latency in detecting lost packets
• Congestion window
– Small value at beginning of connection (e.g., 1 MSS)
– May not reach a high value before transfer is done
• Timeout vs. triple-duplicate ACK
– Two main ways of detecting packet loss
– Timeout is slow, and triple-duplicate ACK is fast
– However, triple-dup-ACK requires many packets in flight
– … which doesn’t happen for very short transfers

McGraw-Hill
103
©The McGraw-Hill Companies, Inc., 2001
 TCP Interaction: Persistent Connections
• Handle multiple transfers per connection
– Maintain the TCP connection across multiple requests
– Either the client or server can tear down the connection
– Added to HTTP after the Web became very popular
• Performance advantages
– Avoid overhead of connection set-up and tear-down
– Allow TCP to learn a more accurate RTT estimate
– Allow the TCP congestion window to increase
• Further enhancement: pipelining
– Send multiple requests one after the other
– … before receiving the first response

McGraw-Hill
104
©The McGraw-Hill Companies, Inc., 2001
 Conclusions
• Key ideas underlying the Web
– Uniform Resource Identifier (URI)
– HyperText Markup Language (HTML)
– HyperText Transfer Protocol (HTTP)
– Browser helper applications based on content type
• Main Web components
– Clients, proxies, and servers
• Dependence on underlying Internet protocols
– DNS and TCP
• Next week: other application-layer protocols
– E-mail, peer-to-peer file sharing, Voice-over-IP

McGraw-Hill
105
©The McGraw-Hill Companies, Inc., 2001
 POP3

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 POP3

• Stands for Post Office Protocol – Version 3

• RFC in 1993
• POP3 “is intended to permit a workstation to
dynamically access a maildrop on a server
host in a useful fashion. Usually, this means
that the POP3 protocol is used to allow a
workstation to retrieve mail that the server is
holding for it.“

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 POP3

• Doesn’t provide extensive mail operations on

the server
• Normally mail is downloaded, then deleted
• Deletion doesn’t really occur until QUIT
• Uses port 110

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Minimal POP3 Commands
• USER name – user’s name
• PASS string – user’s password
• QUIT – quits, and updates the mailbox
• STAT – status
• LIST [msg] – lists the mail messages (#’s only)
• RETR msg – retrieves a mail message
• DELE msg – deletes a message
• NOOP – No operation
• RSET – resets the mail to its original state
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Example POP3 Transaction

Server

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Establish the Socket

Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Server is OK

+OK POP3 server ready

Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Establish the Socket

USER bob
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 This user is OK

+OK
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Send the Password

PASS ITSD4304
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 The Password is OK

+OK
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Ask how the Server is Feeling…

STAT
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 I’m Fine, Thanks!

+OK 2 320
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Got Any Mail?

LIST
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Yup…

+OK 2 messages (320 octets)

Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message One

1 120
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message Two

2 200
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 End

.
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Get Message One

RETR 1
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message One Coming!

+OK 120 octets

Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message One Coming!

<the POP3 server sends message 1>

Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 End of Message 1

.
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Get Message One

DELE 1
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message 1 Gone!

+OK message 1 deleted

Primary Primary
Thread Socket (101) Thread

Server Client

Note: it’s not really gone…

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Get Message One

QUIT
Primary Primary
Thread Socket (101) Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 QUIT – Commits/Updates

+OK bob POP3 server signing off

Primary Primary
Thread Socket (101) Thread

Server Client

Note: now it’s gone

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Socket Disposed

Primary Primary
Thread Thread

Server Client

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 IMAP

(Internet Message Access Protocol)

133
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Content
• about imap
• procedure of the imap protocol
• interaction between client and server
• flags message attributes
• different states of an imap session
– state diagram
• commands
• example of an imap session

134
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 About IMAP
• Internet message access protocol
• Stands for accessing electronic mail that are kept on a
server
• access messages from more than one computer
• More than one user
• Imap protocol based on a TCP connection (port 143)
• No functionality to send messages (realized with
SMTP)

135
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Procedure of the imap protocol

• All emails are kept on a mail server

• Functionality:
– Treat as local directory
– work directly on server
– copy
– delete
– different folders
• download message headers and later on the full
messages

136
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Procedure of the imap protocol [2]

• communication procedure:
– Connection initialization
– Welcome of the server
– Interaction between client and server
– Connection closed

137
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Interaction between client and server

• Client sends data - server receives:

– client-command begins with alphanumeric string
(‘tag’) e.g. A001 which is created for every new
command
– then the command with corresponding arguments
follows

• Server sends data – client receives

– Server receives and works with the command
– Returns the same tag with a state (OK/NO/BAD)
– Additional information begin with ‘*’ (untagged)
– Untagged information can be sent at every time

138
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Different states of an imap session

• Most commands are only valid in special states

(otherwise: protocol error)
• 4 states (initialization is not an own state)
– Non-Authenticated
– Authenticated
– Selected
– Logout, Closed

139
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 State diagram

Initialization
Connect Reject con.

Logou
Non-Authenticated
t

Close
Logout,
Preauth
Login
Logou
Authenticated
t
Select Close
Logou
Selected
t

140
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Flags message attributes
• List of token associated with the message that is sent
by the server
• A flag can be permanent or session-only
• types:
– System flag:
• flag name that is pre-defined in the imap specification
• All system flags begin with ‘\’
– Keywords:
• A keyword is defined by the server implementation
• Keywords do not begin with ‘\’

141
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Flags Message Attributes [2]
• \Seen (Message has been read)
• \Answered (Message has been answered)
• \Flagged (Message is "flagged" for urgent/special
attention)
• \Deleted (Message is "deleted" for removal by later
EXPUNGE)
• \Draft (Message has not completed composition
(marked as a draft))
• \Recent (Message is "recently" arrived in this mailbox.
This session is the first session to have been notified about
this message)

142
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 commands
• LOGIN username password
• SELECT mailbox-name
• CREATE new-mailbox-name
• DELETE mailbox-name
• RENAME old-name new-name
• AUTHENTICATE
• CLOSE
• SEARCH
• COPY

143
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 imap session
S: * OK IMAP4rev1 Service Ready
C: a001 login TIM timmy
S: a001 OK LOGIN completed
C: a002 select inbox
S: * 18 EXISTS
S: * FLAGS(\Answered\Flagged\Deleted\Seen\Draft)
S: * 2 RECENT
S: * OK [UNSEEN 17] Message 17 is the first unseen
message
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: a002 OK [READ-WRITE] SELECT completed
C: a003 fetch 12 body[header]
S: * 12 FETCH (BODY[HEADER] {350}
S: Date: Wed, 19 May 2003 02:23:25 -0700 (PDT)
S: From: christiane <christiane@[Link]>
S: Subject: weekend planning

144
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 imap session [2]
S: To: steffen <steffen@[Link]>
S: cc: steffen <steffen@[Link]>
S: Message-Id: <B27397-0100000@[Link]>
S: MIME-Version: 1.0
S: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
S:
S: )
S: a003 OK FETCH completed
C: a004 store 12 +flags \deleted
S: * 12 FETCH (FLAGS (\Seen \Deleted))
S: a004 OK +FLAGS completed
C: a005 logout
S: * BYE IMAP4rev1 server terminating connection
S: a005 OK LOGOUT completed

145
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Simple Mail Transfer Protocol

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Overview

• Introduction
• Message Breakdown
• Sending Messages
• Extensions
• Retrieving mail (Mailbox Protocols)
• Examples

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Email Statistics

Email Statistics (2004):

• 31 billion emails sent daily,
expected to double by 2006
• Email generates about one
billion Gigabytes of new
“information” per year
• Spam accounts for about 40%
of all email traffic

[Link]

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SMTP
• Protocol originated in 1982 (RFC821, Jon Postel)
• Standard message format (RFC822,2822, D. Crocker)
• Goal: To transfer mail reliably and efficiently

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SMTP
• SMTP clients and servers
have two main components

– User Agents – Prepares the

message, encloses it in an
envelope. (ex. Thunderbird,
Eudora)

– Mail Transfer Agent –

Transfers the mail across the
internet (ex. Sendmail,
Exim)

– Analogous to the postal

system in many ways

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SMTP
• SMTP also allows the
use of Relays
allowing other MTAs
to relay the mail

• Mail Gateways are

used to relay mail
prepared by a protocol
other than SMTP and
convert it to SMTP

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Format of an email
• Mail is a text file
• Envelope –
– sender address
– receiver address
– other information
• Message –
– Mail Header – defines
the sender, the receiver,
the subject of the
message, and other
information
– Mail Body – Contains
the actual information in
the message

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

Post Office From kissel@[Link] Tue Oct 25 20:27:21 2005
Return-Path: <kissel@[Link]>
X-Original-To: kissel@[Link]
Delivered-To: kissel@[Link]
Mailbox Received: by [Link] (Postfix, from userid 62)
id 8EC8D18D; Tue, 25 Oct 2005 20:27:21 -0400 (EDT)
Received: from [Link] ([Link]
[[Link]])

Post office by [Link] (Postfix) with ESMTP id 59888C9

for <kissel@[Link]>; Tue, 25 Oct 2005 20:27:20 -0400 (EDT)
Received: by [Link] (Postfix, from userid 62)
id 344482045; Tue, 25 Oct 2005 20:27:20 -0400 (EDT)
and mail Received: from [Link] ([Link] [[Link]])
by [Link] (Postfix) with ESMTP id 3932E1ECA

route
for <kissel@[Link]>; Tue, 25 Oct 2005 20:27:19 -0400 (EDT)
Date: Tue, 25 Oct 2005 20:27:19 -0400 (EDT)

From: Ezra Kissel <kissel@[Link]>

X-X-Sender: kissel@[Link]
To: kissel@[Link]
Subject: email test
Message-ID: <[Link].4.62.0510252026550.4176@[Link]>
X-Sanitizer: This message has been sanitized!
X-Sanitizer-URL: [Link]
X-Sanitizer-Rev: UDEL-ECECIS: [Link],v 1.64 2002/10/22 MIME-Version: 1.0
X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on [Link]
Receiver’s X-Spam-Level:
X-Spam-Status: No, score=-3.8 required=4.1 tests=ALL_TRUSTED,BAYES_00
autolearn=ham version=3.0.4
X-Sanitizer: This message has been sanitized!
Mailbox X-Sanitizer-URL: [Link]
X-Sanitizer-Rev: UDEL-ECECIS: [Link],v 1.64 2002/10/22 MIME-Version: 1.0
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset="US-ASCII"; format=flowed
Status: RO
X-Status:
X-Keywords:
X-UID: 50

This is a test message.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 How SMTP works (A-PDU’s)
• The Basics

Keyword Arguments
HELO Sender’s Host Domain
Name
MAIL Email Address of sender
FROM:
RCPT TO: Email of Intended
recipient
DATA Body of the message
McGraw-Hill
QUIT ©The McGraw-Hill Companies, Inc., 2001
 How SMTP works (A-PDU’s)
• The Extras

Keyword Arguments
RSET
VRFY Name to be verified
NOOP
TURN
EXPN Mailing list to expand
HELP Command Name

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Status Codes

• The Server responds with a 3 digit code that

may be followed by text info
– 2## - Success
– 3## - Command can be accepted with
more information
– 4## - Command was rejected, but error
condition is temporary
– 5## - Command rejected, Bad User!

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Connection Establishment

TCP Connection
Establishment

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Message Progress

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Connection Termination

TCP Connection Termination

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Limitations in SMTP

• Only uses NVT 7 bit ASCII format

– How to represent other data types?

• No authentication mechanisms
• Messages are sent un-encrypted
• Susceptible to misuse (Spamming,
faking sender address)

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Solution: SMTP extensions
• MIME – Multipurpose Internet Mail Extensions
– Transforms non-ASCII data to NVT (Network Virtual Terminal)
ASCII data
• Text
• Application
• Image
• Audio
• Video

RFC 1425, 1426, 1521

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Email Systems

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Overview

• Email Basics
• What Makes Up An Email
• How Email Works
• What Are TCP/IP Ports

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Email Basics

• What is an Email – an electronic message

transmitted over a network from one user to
another.
• Can be as simple as a few lines of text, or
include attachments such as pictures or
documents.
• Email made up 75% of network traffic soon
after the introduction of the internet.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 What Makes Up An Email

• The Header
– Who sent the email.
– To whom the mail is sent.
– When the email was sent.
– The email subject.
– The size of the email.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 What Makes Up An Email

• The Body
– Contains the message.
– May also contain an attachment.
• Attachments
– If not embedded within the body, attachments are
sent along with the email.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works

• Different Architectural Models exist for

constructing computer systems.
• Some models include:
– Peer-Peer
– Pipe and Filter
– Implicit Invocation
– Client-Server

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Peer-Peer Model

Provided Interface Provided Interface

Peer Peer
Required Interface Required Interface

Provided Interface

Peer
Required Interface

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Provided Interface

Filter

McGraw-Hill
Required Interface

Provided Interface

Filter
Required Interface

Provided Interface

Filter
Required Interface
How Email Works
Pipe and Filter Model

Provided Interface

Filter
Required Interface
©The McGraw-Hill Companies, Inc., 2001
 How Email Works
Implicit Invocation “Bus” Model

Provided Interface Provided Interface Provided Interface Provided Interface

Component Component Component Component

Required Interface Required Interface Required Interface Required Interface

Provided Interface

Bus
Required Interface

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Client-Server Model
Provided Interface Provided Interface Provided Interface

Client Client Client

Required Interface Required Interface Required Interface

Provided Interface

Server
Required Interface

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works

• The model that works best for email is the

Client-Server model.
• Clients carry out user interactions with the
email server.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Clients
• Forms in which clients appear:
– Application based - these are installed onto user’s
machines and include Microsoft Outlook and the
freely available Outlook Express and Eudora.
– Web based - these appear in a web browser’s
window and include Hotmail, Yahoo and Outlook
web client.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Clients
• Clients vary greatly in functionality, but all provide a
basic level of functionality that assists the user.
• Basic functions include:
– Ability to create new emails.
– Display and store received emails.
– Hold address lists of contacts, a calendar, journal and other
extra functions that help organize the user’s working day.
– The client is also configured with the account information
and names or IP addresses of the email servers with which
it will be communicating.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Servers
• An email server is typically a combination of processes
running on a server with a large storage capacity – a list of
users and rules, and the capability to receive, send and store
emails and attachments.
• These servers are designed to operate without constant user
intervention.
• Should process emails for months as sending, receiving and
maintenance tasks are carried out at scheduled times. The
client only has to connect to the email server when it sends
and checks/receives new email.
• Sometimes it may be permanently connected to the server to
allow access to shared address books or calendar information
– this is typical of a LAN-based email server.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Servers
• Most email servers conduct email services by running
two separate processes on the same machine.
• One process is the POP3 (Post Office protocol 3)
server, which holds emails in a queue and delivers
emails to the client when they are requested.
• The other is the SMTP (simple mail transfer protocol)
server that receives outgoing emails from clients and
sends and receives email from other SMTP servers.
• These two processes are linked by an internal mail
delivery mechanism that moves mail between the
POP3 and SMTP servers.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Servers
• When the client calls the email server to send or
check for mail it connects to the server on certain
TCP/IP ports:
– SMTP on port 25
– POP3 on port 110.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 How Email Works
Servers
• Email systems come in various formats, but
the most common rely on a single server that
provides both POP3 and SMTP services.
• Sometimes, in large organizations, these
services are separated onto different servers.
• Currently, the majority of email servers use
Windows NT or 2000.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 What Are TCP/IP Ports
• Most email servers run on a web server platform with
email services installed.
• Each server has one or more unique TCP/IP
(transmission control protocol/internet protocol)
addresses. Attached to all TCP/IP addresses are many
ports that range from 0 to 65,535.
• TCP/IP uses ports to allocate different jobs to
different services. The server will listen for a client or
application to call it on a port and direct traffic from
that port to the required service.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIME Headers
• Located between the Email Header and Body

– MIME-Version: 1.1
– Content-Type: type/subtype
– Content-Transfer-Encoding: encoding type
– Content-Id: message id
– Content-Description: textual explanation of non-
textual contents

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIME Headers (cont’d)
• Content-Type – Type of data used in the Body
– Text: plain, unformatted text; HTML
– Multipart: Body contains different data types
– Message: Body contains a whole, part, or pointer to a message
– Image: Message contains a static image (JPEG, GIF)
– Video: Message contains an animated image (MPEG)
– Audio: Message contains a basic sound sample (8kHz)
– Application: Message is of data type not previously defined

• Content-Transfer-Encoding – How to encode the message

– 7 bit – no encoding needed
– 8 bit – Non-ASCII, short lines
– Binary – Non-ASCII, unlimited length lines
– Base64 – 6 bit blocks encoded into 8-bit ASCII
– Quoted-printable – send non-ASCII characters as 3 ASCII characters, =##, ## is the
hex representation of the byte

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Base64 Encoding

• Divides binary data into 24 bit blocks

• Each block is then divided into 6 bit chunks
• Each 6-bit section is interpreted as one character,
25% overhead

11001100 10000001 00111001

110011 001000 000100 111001

(51) (8) (4) (57)
(z) (I) (E) (5)
01111010 01001001 01000101 00110101
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Quoted-Printable Encoding

• Used when the data has a small non-ASCII portion

• Non-ASCII characters are sent as 3 characters
• First is ‘=‘, second and third are the hex representation of
the byte

01001100 10011101 00111001

(=) (9) (D)

00111101 00111001 01000100

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Multipart, Encoded MIME Message

From: Ezra Kissel <kissel@[Link]>

To: kissel@[Link]
Subject: attachment test
MIME-Version: 1.0
Content-Type: MULTIPART/MIXED; boundary="MIMEStream=_0+92061_793033260215529_597673089"

--MIMEStream=_0+92061_793033260215529_597673089
Content-Type: TEXT/PLAIN; charset="US-ASCII"; format=flowed

There is an image attached...

--MIMEStream=_0+92061_793033260215529_597673089
Content-Type: IMAGE/jpeg; name="[Link]"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="[Link]"

/9j/4AAQSkZJRgABAgEASABIAAD/4QNxRXhpZgAATU0AKgAAAAgABwESAAMA
AAABAAEAAAEaAAUAAAABAAAAYgEbAAUAAAABAAAAagEoAAMAAAABAAIAAAEx
…
osv/0I5nPvr7sVdirsVf/9GO5z76+6hxQ1il2Kv/0o5nPvr7eKtYq7FX/9k=

--MIMEStream=_0+92061_793033260215529_597673089--
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 MTA MTA
SMTP AUTH Client TCP Connection
Establishment
Server

• Allows the server to provide features 220 Service Ready

only to known users and limit others.
EHLO [Link]

• Various authentication methods may 250 Hello [Link]

be used (PLAIN, LOGIN, CRAM- Negotiate Encryption

MD5, etc.) (STARTTLS)

AUTH PLAIN AGV6cmEAYg==

• Encryption is highly recommended if
not enforced by MTA. 235 Authentication Succeeded

MAIL FROM: kissel@[Link]

• Ex. AUTH PLAIN
– Simple 250 OK

– Usage: AUTH PLAIN <id>\ RCPT TO: amer@[Link]

0<user>\0<password>
250 OK
– Authentication string is Base64
encoded
RFC 1869, 2554, 2595
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Email can be faked…
HELO [Link]
MAIL FROM: cis-dept@[Link]
RCPT TO: amer@[Link]
DATA Solutions
From: Department Chair
To: Dr. Paul Amer
• Email signatures (PGP)
Subject: CISC856 • Sender Policy Framework
(SPF)
Dr. Amer,
By department decree all students in your CISC856 TCP/IP
class are hereby to be given automatic A’s.
Thank you,
Department Chair
.
QUIT
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 MTAs and Mail Access Protocols

• The MTA delivers email to the user’s mailbox

• Can be complex with numerous delivery methods,
routers, and ACLs
• Exim, Postfix, Sendmail

• The Mail Access Protocols are used by the users

to retrieve the email from the mailbox
– POP3
– IMAP4

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 POP vs. IMAP
POP3:
All Messages
Internet
Whole
messag
e

IMAP:
[Link]
Friends
…. rs
heade
Internet

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Post Office Protocol v3

• Simple
• Allows the user to obtain a list of their Emails
• Users can retrieve their emails
• Users can either delete or keep the email on
their system
• Minimizes server resources

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Internet Mail Access Protocol v4
• Has more features than POP3
• User can check the email header before downloading
• Emails can be accessed from any location
• Can search the email for a specific string of
characters before downloading
• User can download parts of an email
• User can create, delete, or rename mailboxes on a
server

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Multipurpose Internet Mail Extensions
(MIME)

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIME

• Important RFCs
– RFC-822 Standard for the format for ARPA Internet test messages
– RFC-2045 MIME Part 1: Format of Internet Message Bodies
– RFC-2046 MIME Part 2: Media Types
– RFC-2047 MIME Part 3: Message Header Extensions
– RFC-2048 MIME Part 4: Registration Procedure
– RFC-2049 MIME Part 5: Conformance Criteria

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIME – What is it?

• MIME refers to an official Internet standard that specifies

how messages must be formatted so that they can be
exchanged between different email systems.
• MIME permits the inclusion of virtually any type of file
or document in an email message.
• Specifically, MIME messages can contain
– text
– images
– audio
– video
– application-specific data.
• spreadsheets
• word processing documets

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIME Features

• Support of character sets other than ASCII

• Content type labeling System
• Support of non-text content in e-mail messages
• Support for compound documents

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Non-ASCII Character set support
• Message header
– content-type field
• put in the header by the client program creating the e-mail for use by the client
program used to display the received message
• charset= optional parameter
– if absent ASCII is assumed

• Content-Type: text/plain; charset="ISO-8859-1"

– ISO-8859-1 extends the basic character set of ASCII to include many of the
accented characters used in languages such as Spanish, French, and German.
– US-ASCII is the standard character set used in the US

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Content Labeling

• a set of registered MIME Types that map to specific file types

– MIME Types consist of :
• a primary type
• a sub type separated by a / ( as text/html)
• Common Mime Types:
FileExtension MIME Type Description
.txt text/plain Plain text
.htm text/html Styled text in HTML format
.jpg image/jpeg Picture in JPEG format
.gif image/gif Picture in GIF format
.wav audio/x-wave Sound in WAVE format
.mp3 audio/mpeg Music in MP3 format
.mpg video/mpeg Video in MPEG format
.zip application/zip Compressed file in PK-ZIP format
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Non-text Content

• To be sent through the e-mail system non-textual content must be

converted (encoded) to ASCII for transmission and unencode
back to its original format for display upon receipt.
– originally done via uuencode
– MIME uses base 64 encoding (RFC 2045)
• binary to text encoding scheme
• targets A-Z, a-z,0-9, +,/
• scheme:
– take three byte of data, put into a 24 bit buffer
– extract 4 six bit values
– use each value as an index into:
» ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
– this yields 4 ASCII characters
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 base64 encoding example

Man is distinguished, not only by his reason, but by this singular passion from
other animals, which is a lust of the mind, that by a perseverance of delight in the
continued and indefatigable generation of knowledge, exceeds the short
vehemence of any carnal pleasure.

base64 encoded:
TWFuIGlzIGRpc3Rpbmd1aXNoZWQsIG5vdCBvbmx5IGJ5IGhpcyByZWFzb24sIGJ1dCBieSB0
aGlzIHNpbmd1bGFyIHBhc3Npb24gZnJvbSBvdGhlciBhbmltYWxzLCB3aGljaCBpcyBhIGx1
c3Qgb2YgdGhlIG1pbmQsIHRoYXQgYnkgYSBwZXJzZXZlcmFuY2Ugb2YgZGVsaWdodCBpbiB0
aGUgY29udGludWVkIGFuZCBpbmRlZmF0aWdhYmxlIGdlbmVyYXRpb24gb2Yga25vd2xlZGdl
LCBleGNlZWRzIHRoZSBzaG9ydCB2ZWhlbWVuY2Ugb2YgYW55IGNhcm5hbCBwbGVhc3VyZS4=

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Multipart Messages
• use content-type = multipart/sub type
– sub types :
• related
• mixed
• see examples at [Link]

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SNMP
Simple Network Management
Protocol

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Simple Network Management
•
Protocol
SNMP is a framework that provides facilities for managing and monitoring
network resources on the Internet.
• Components of SNMP:
– SNMP agents
– SNMP managers
– Management Information Bases (MIBs)
– SNMP protocol itself
SNMP agent
SNMP
manager

SNMP agent
SNMP
protocol
messages
SNMP agent

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Simple Network Management
• Protocol
SNMP agent is software that runs on a piece of network
equipment (host, router, printer, or others) and that
maintains information about its configuration and current
state in a database
• Information in the database is described by Management
Information Bases (MIBs)
• An SNMP manager is an application program that
contacts an SNMP agent to query or modify the database
at the agent.
• SNMP protocol is the application layer protocol used by
SNMP agents and managers to send and receive data.
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 SNMP

• Interactions in SNMP
Management Station Managed System

SNMP Manager SNMP Agent

Process Process MIB
Queries

Replies

Access
objects
Traps

Traps
Data
SNMP SNMP messages SNMP
UDP UDP
IP IP

IP Network

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 MIBS
• A MIB specifies the managed objects
• MIB is a text file that describes managed objects using the
syntax of ASN.1 (Abstract Syntax Notation 1)
• ASN.1 is a formal language for describing data and its
properties

• In Linux, MIB files are in the directory

/usr/share/snmp/mibs
– Multiple MIB files
– MIB-II (defined in RFC 1213) defines the managed objects of
TCP/IP networks

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Managed Objects
• Each managed object is assigned an object identifier (OID)
• The OID is specified in a MIB file.
• An OID can be represented as a sequence of integers
separated by decimal points or by a text string:
Example:
– [Link].[Link].
– [Link]

• When an SNMP manager requests an object, it sends the

OID to the SNMP agent.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Organization of managed objects
• Managed objects are
organized in a tree-like . root

hierarchy and the OIDs iso(1)

reflect the structure of the
hierarchy. org (3)

• Each OID represents a node

dod (6)
in the tree.
• The OID [Link].2.1 internet (1)

([Link]
directory (1) mgmt (2) experimental (3) private (4)
b-2) is at the top of the
hierarchy for all managed
objects of the MIB-II. mib-2 (1)

• Manufacturers of networking
system (1)
equipment can add product at (3) icmp (5) udp (7) snmp (11)

specific objects to the interface (2) ip (4) tcp (6) egp (8) transmiss
hierarchy.
ipForwDatagrams (6)

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Definition of managed objects in a
MIB
• Specification of ipForwDatagrams in MIB-II.

ipForwDatagrams OBJECT-TYPE
SYNTAX Counter
ACCESS read-only
STATUS mandatory
DESCRIPTION
"The number of input datagrams for which this
entity was not their final IP destination, as a
result of which an attempt was made to find a
route to forward them to that final destination.
In entities which do not act as IP Gateways, this
counter will include only those packets which were
Source-Routed via this entity, and the Source-
Route option processing was successful."
::= { ip 6 }

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SNMP Protocol
• SNMP manager and an SNMP agent
communicate using the SNMP protocol
– Generally: Manager sends queries and agent
responds
– Exception: Traps are initiated by agent.
get-request
get-response Port 161

SNMP manager
get-next-request
SNMP agent
get-response Port 161

set-request
get-response Port 161

Port 162 trap

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SNMP Protocol
• Get-request. Requests the values of one or more objects
• Get-next-request. Requests the value of the next object,
according to a lexicographical ordering of OIDs.
• Set-request. A request to modify the value of one or more
objects
• Get-response. Sent by SNMP agent in response to a get-
request, get-next-request, or set-request message.
• Trap. An SNMP trap is a notification sent by an SNMP agent
to an SNMP manager, which is triggered by certain events at
the agent.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Traps
• Traps are messages that asynchronously sent by an
agent to a manager
• Traps are triggered by an event
• Defined traps include:
– linkDown: Even that an interface went donw
– coldStart - unexpected restart (i.e., system crash)
– warmStart - soft reboot
– linkUp - the opposite of linkDown
– (SNMP) AuthenticationFailure
– …

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SNMP Versions
• Three versions are in use today:
– SNMPv1 (1990)
– SNMPv2c (1996)
• Adds “GetBulk” function and some new types
• Adds RMON (remote monitoring) capability

– SNMPv3 (2002)
• SNMPv3 started from SNMPv1 (and not SNMPv2c)
• Addresses security

• All versions are still used today

• Many SNMP agents and managers support all three
versions of the protocol.
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 Format of SNMP Packets

• SNMPv1 Get/Set messages:

Version Community SNMP PDU

Cleartext string that is

used as a password
PDU Type Request ID
PDU type, e.g.:
32: SNMPv1 Get Error Status Error Index
64: SNMPv2 Get
Object 1, Value 1

Unique ID to match Object 2, Value 2

requests with replies
...
Sequence of name-value
pairs
McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001
 SNMP Security
• SNMPv1 uses plain text community strings for
authentication as plain text without encryption

• SNMPv2 was supposed to fix security problems, but

effort de-railed (The “c” in SNMPv2c stands for
“community”).
•
• SNMPv3 has numerous security features:
– Ensure that a packet has not been tampered with (integrity),
– Ensures that a message is from a valid source (authentication)
– Ensures that a message cannot be read by unauthorized (privacy).

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 SNMP Security

• Security model of SNMPv3 has two components:

[Link] of granting access rights to a community,
SNMPv3 grants access to users.
2. Access can be restricted to sections of the MIB
(Version-based Access Control Module (VACM).
Access rights can be limited
• by specifying a range of valid IP addresses for a user or
community,
• or by specifying the part of the MIB tree that can be
accessed.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001

 Security levels in SNMPv2
SNMP has three security levels:
• noAuthNoPriv: Authentication with matching a user name.
• authNoPriv: Authentication with MD5 or SHA message
digests.
• authPriv: Authentication with MD5 or SHA message
digests, and encryption with DES encryption

Compare this to SNMPv1 and SNMPv2c:

• SNMPv1, SNMPv2: Authentication with matching a
community string.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2001