E-Mail System
lctseng (2020-2023, CC-BY)
? (?-2019)
國立陽明交通大學資工系資訊中心
Computer Center of Department of Computer Science, NYCU
1
Components of an E-Mail (1)
● You can really see …
○ Headers, which can be forged, altered, etc.
○ Body
Date: Thu, 30 Mar 2023 09:15:04 +0800 (CST)
From: NCTU CSCC Help <help@[Link]>
The Header =>
To: lctseng@[Link]
Subject: [CSCC] Test Mail
Blank Line =>
The Body => This is a test mail.
2
Components of an E-Mail (2)
From: Hacker
● Three major components
○ The envelope
To: Victim
■ Invisible to users
■ Determine where the message should be delivered, or to
Mail systems rely on this
whom it should be returned
○ The headers Dear Bob:
■ Information about the messages, defined in RFC2822 The mail
● Date, From, To, Content-Type, charset body goes
● Content-Length, MessageID, … here...
● No checking consistent "To" in envelope and header
By Alice
○ The message body
Can be anything!
■ Text, attachments, ...
3
Mail System
● Major components ○ Delivery Agent (DA)
○ Mail User Agent (MUA) ■ Place mails in users' mailboxes
■ Help user read and compose mails ○ Access Agent (AA)
○ Submission Agent (SA) ■ Connect the user agents to the
■ Route mails to local MTA mailboxes using POP3 or IMAP
○ Mail Transport Agent (MTA) protocols
■ Route mails among machines
HostA - sender HostB - receiver
UA DA
Web Mail sendmail
TA TA Message UA = User agent
Postfix Postfix
(port 25)
store SA = Submission agent
UA DA TA = Transport agent
mail Sieve DA = Delivery agent
SA
Internet
AA = Access agent
Postfix
(port 587)
UA UA AA
mutt Thunderbird Dovecot
4
Mail System – The User Agent
● Help user read and compose mails
○ UA must know mail format
■ Previously: Text only (command line agents)
■ Now: MIME
※ MIME (Multipurpose Internet Mail Extensions)
○ Include several types of content that can be encoded in the mail
■ 7-bit base64, 8-bit binary, ...
■ image, video, virus, …
5
Mail System – The Submission Agent
● Route mails to local MTA
○ Typical works that a MTA must do:
■ Ensuring that all hostname are fully qualified
■ Modifying headers
● MessageID
● Date
● DomainKeys/DKIM
■ Logging errors
■ …
○ RFC2476 introduces the idea of splitting
MTA
■ Let SA to share the load
6
Mail System – The Transport Agent (1)
● Route mails among machines
○ Accept mail from UA, examine the recipients' addresses, and
delivery the mail to the correct host
○ Protocols
■ SMTP (Simple Mail Transport Protocol)
● RFC 821 (1982)
■ ESMTP (Extended SMTP)
● RFC 2821 (2001) => 5321 (2008) => 7504 (2015)
○ Popular transport agents
■ sendmail [Link]
■ Postfix [Link]
■ exim, qmail, …
7
Mail System – The Transport Agent (2)
● Conversation between MTAs
○ Threat of eavesdropping
Local Remote
hello
hello
mail from sender
OK
mail to friend@remote
OK
Here comes the message
OK
Done
OK
8
Mail System – The Transport Agent (3)
● Protocol: SMTP
$ telnet [Link] 25 mail from: <alice@[Link]>
Trying [Link]... 250 2.1.0 Ok
Connected to [Link]. rcpt to: <bob@[Link]>
Escape character is '^]'. 250 2.1.5 Ok
220 [Link] ESMTP Postfix data
ehlo [Link] 354 End data with <CR><LF>.<CR><LF>
[Link] From: haha <devnull@[Link]>
250-PIPELINING To: admin@[Link]
250-SIZE 204800000
250-VRFY hehe... I spammed you!
250-ETRN .
250-ENHANCEDSTATUSCODES 250 2.0.0 Ok: queued as 81BD4FB4
250-8BITMIME quit
250 DSN 221 2.0.0 Bye
Connection closed by foreign host.
From: haha <devnull@[Link]>
To: admin@[Link]
Message-Id: <20230330070002.81BD4FB4@[Link]>
Date: Thu, 30 Mar 2023 14:59:53 +0800 (CST)
hehe... I spammed you! 9
Mail System – The Delivery Agent
● Place mails in users' mailboxes
○ Accept mail from MTA and deliver the mail to the local recipients
○ Type of recipients
■ User
■ Program
● Sieve filters, procmail (deprecated), ...
○ Sieve - mail filtering language (RFC 5228)
■ Many implementations
■ Pigeonhole - Sieve implementation
provided by Dovecot
● Official documentation
10
Mail Storage
● The place on the local machine where email is stored
○ Usually the directory: /var/mail or /var/spool/mail
■ Users' mails are stored in files named with each user's login name
● Eg. /var/mail/lctseng
■ Permission "775" and root:mail as the owner and group owner
● drwxrwxr-x 2 root mail 512 Dec 16 15:51 mail/
○ Using database
■ When the organization is large or for ISP with millions of customers
■ Easy to search, categorize
11
Mail System – The Access Agent
● Help user download mail from server
○ Protocols
■ IMAP (Internet Message Access Protocol)
■ POP3 (Post Office Protocol – Version 3)
12
Mail Addressing – Domain (1)
● Two kinds of email addresses:
○ Route based address (obsolete)
■ Message will travel through several intermediate hosts to the
destination
■ Format: host!path!user
● E.g. sender!path!to!destination!recipient
● This mail is sent from "sender" host to the user "recipient" at
"destination" host
○ Location independent address (relies on DNS)
■ Simply identify the final destination
■ Format: user@[Link]
● E.g. ta@[Link]
13
Mail Addressing – Domain (2)
● Where to send the mail?
○ When you want to send a mail to lctseng@[Link], the MTA
will:
□ First, lookup up the mail exchanger of "[Link]"
$ dig mx [Link]
;; ANSWER SECTION:
[Link]. 3600 IN MX 5 [Link].
[Link]. 3600 IN MX 10 [Link].
[Link]. 3600 IN MX 5 [Link].
□ If there is any servers, try from servers with higher priority (smaller
value)
□ If no MX records, mail it directly to the host (A record)
14
Mail Addressing – Domain (3)
● Why using "Mail eXchanger"?
○ Centralize all the mail tasks to group of servers
■ Security enforcement, firewall control, …
○ More robust
■ load balancing, fail over, …
15
Mail Addressing – Alias
● Alias
○ Map a username to something else
■ Mailing list
■ Be careful of mail looping
● Several mechanisms to define aliases:
○ Traditional method: in files
○ Traditional method + NIS
○ LDAP (Light-weight Directory Access Protocol)
● When the mail server wants to resolve name
○ File-based method
○ LDAP-based method
16
Mail Alias – Mechanisms (1)
● Places for defining alias
○ [sender] In configuration file of a MUA
■ MUA expands the alias before injecting the message into the mail
system
○ [receiver] In the system-wide /etc/mail/aliases file
■ Read by MDA
■ Deliver to the new destination when receiving the mail
○ [receiver] In user's forwarding file, ~/.forward
■ Read by MDA after system-wide alias file
■ forward(5)
17
Mail Alias – Mechanisms (2)
● The format of an entry in aliases file
1. Local-name: recipient1,recipient2,…
■ E.g.
● admin: lwhsu,wangth,jnlin
● lctseng: lctseng@[Link]
● root: ta
2. Local-name: :include:filename
■ E.g. lwhsu
● ta: :include:/usr/local/mail/TA fyli
lctseng
jnlin
wangth
pmli
Contents of TA
18
Mail Alias – Mechanisms (3)
● The format of an entry in aliases file
3. Local-name: absolute-path-file
■ Appended to file
■ Ex:
● complaints: /dev/null
● troubles: trouble_admin,trouble_log
● trouble_admin: :include:/usr/local/mail/troadm
● trouble_log: /usr/local/mail/logs/troublemail
4. Local-name: "|program-path"
■ Inject as STDIN
■ Ex:
● autoftp: "|/usr/local/bin/ftpserver"
● nahw3: "|/home/nahw3/[Link]"
19
Mail Alias – Mechanisms (4)
● The hashed aliases DB
○ /etc/mail/aliases: plain text aliases information
○ /etc/mail/[Link]: hashed version for efficiency
○ "newaliases" command
■ Rebuild the hashed version when changing the aliases file
■ Files read from ":include:" is outside the aliases file
20
Mail Alias – Mechanisms (5)
● User maintainable forwarding file
○ ~/.forward
○ Format: comma-separated
○ E.g.
■ lctseng@[Link]
■ \lctseng, lctseng@[Link], lctseng@[Link]
○ backslash + username
■ Bypassing further redirection (deliver to mailbox directly)
○ Must be owned by user and with permission of 600
■ The path to .forward file should be writable only to user
21
Mail Alias – Mechanisms (6)
● Alias must
○ postmaster and MAILER-DAEMON
■ Mail system maintainer MAILER-DAEMON: postmaster
postmaster: root
○ bin, sys, daemon, nobody, … bin: root
bind: root
■ System accounts (root) daemon:root
games: root
○ root kmem: root
mailnull: postmaster
■ forward root mail to the administrator nobody:root
● /root/.forward operator: root
…
● aliases
22
Mail Headers (1)
● Defined by RFC2822
○ Mail reader will hide some uninteresting header information
Date: Wed, 18 Apr 2007 14:05:04 +0800
From: 大小姐 <lkkg-girl@[Link]>
Subject: 笑狗好可怕
To: Yung-Hsiang Liu <liuyh@[Link]>
User-Agent: Mutt/1.5.15 (2007-04-06)
你趕快把牠趕跑好不好？
23
Mail Headers (2)
From chwong@[Link] Wed Apr 18 14:07:21 2007
Return-Path: <chwong@[Link]>
X-Original-To: liuyh@[Link]
Delivered-To: liuyh@[Link]
Received: from [Link] ([Link] [[Link]])
by [Link] (Postfix) with ESMTP id 22EC73B4D51
for <chwong@[Link]>; Wed, 18 Apr 2007 14:07:21 +0800 (CST)
Received: from [Link] (localhost [[Link]])
by [Link] (8.13.8/8.13.8) with ESMTP id l3I654P3060925
for <chwong@[Link]>; Wed, 18 Apr 2007 14:05:04 +0800 (CST)
(envelope-from chwong@[Link])
Received: (from chwong@localhost)
by [Link] (8.13.8/8.13.8/Submit) id l3I654AY060924
for chwong@[Link]; Wed, 18 Apr 2007 14:05:04 +0800 (CST)
(envelope-from chwong)
Date: Wed, 18 Apr 2007 14:05:04 +0800
From: =?utf-8?B?5aSn5bCP5aeQ?= <lkkg-girl@[Link]>
To: Yung-Hsiang Liu <liuyh@[Link]>
Subject: =?utf-8?B?56yR54uX5aW95Y+v5oCV?=
Message-ID: <20070418060503.GA60903@[Link]>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
User-Agent: Mutt/1.5.15 (2007-04-06)
Status: RO
Content-Length: 23
Lines: 1
你趕快把牠趕跑好不好？
24
Mail Transport Example (1)
● User eric@[Link] sends a email to user
evi@[Link]
○ $ dig mx [Link]
■ [Link]
25
Mail Transport Example (2)
● Headers in the example
○ From eric@[Link]
■ Added by [Link] when the mail is put in user's mailbox
■ Used to separate message boundary
○ Return-Path: eric@[Link]
■ The envelope "mail from"
■ Used to send the error message to this address
■ May be different to the "From" address in usual header
○ Delivered-To: evi@rupertsberg
■ Final envelope "rcpt to"
26
Mail Transport Example (3)
● Headers in the example
○ Received: from [Link] (localhost [[Link]]) by [Link]
(8.9.3/8.9.2) with ESMTP id GAA18984; Fri 1 Oct 1999 06:04:02 -800 (PST)
■ Every machine that is ever processed this mail will add a "Received" record in top of
headers
● Sending machine
● Receiving machine
● Mail server software in receiving machine
● Unique queue identifier of mail server in receiving machine
● Date and time
27
Mail Transport Example (4)
● Received: from [Link] (root@[Link] [[Link]]) by
[Link] (8.9.3/8.9.2) with ESMTP id HAA21741 for <evi@[Link]>;
Fri, 1 Oct 1999 07:04:25 -0700 (MST)
● Received: from [Link] ([Link] [[Link]]) by [Link]
(8.9.3/8.9.2) with ESMTP id HAA26176 for <evi@[Link]>; Fri, 1 Oct 1999 07:04:24 -0700
(MST)
● Received: from [Link] ([Link] [[Link]]) by [Link]
(8.9.3/8.9.2) with ESMTP id HAA09899 fro <evi@[Link]>; Fri, 1 Oct 1999 07:04:23 -700
(MST)
● Received: from [Link] (localhost [[Link]]) by [Link] (8.9.3/8.9.2) with
ESMTP id GAA18984; Fri 1 Oct 1999 06:04:02 -800 (PST)
28
Mail Transport Example (5)
● Message-Id: <199910011404.GAA18984@[Link])
○ Add by sender's MTA
● X-Mailer: exmh version 2.0.2 2/24/98
○ MUA
○ Non-standard header information
● To: Evi Nemeth <evi@[Link]>
● Subject: Re: hi
● Date: Fri, 1 Oct 1999 06:04:02 -800
29
Mail System Architecture
● Components in a mail system architecture
○ Mail servers for incoming and/or outgoing mails
○ Storage for mailboxes
○ IMAP or POP3 to integrate PC and remote clients
● Simplest architecture
○ Only one machine
■ MTA server for SMTP (Postfix)
■ Local disk storage for mailboxes
■ MAA server for IMAP/POP3 (Dovecot)
30
Mail System Architecture –
Scalable architecture for medium sites
● Centralize
○ At least one machine for incoming message and
■ Mail home can be the same host or another one
○ At least one machine for outgoing message
■ Each host run MSA and forward mail to the same mail-out server or send
the mail directly
31
To, CC, and BCC
● You should always make sure you mail the right people
○ The To field is for people that the message directly affects, and that
you require actions from.
○ The CC (or Carbon Copy) field is for people you want to know
about the message, but are not directly involved.
○ The BCC field (Blind Carbon Copy) is used when you want other
people to receive the message, but you don't want the other
recipients to know they got it.
● There are "To" and "CC," but not "BCC" in the email headers.
○ Why "No checking consistent 'To' in envelope and header"
32
vacation
● vacation(1): E-mail auto-responder
○ returns a message, ~/.[Link] by default
○ ~/.[Link]
■ default database file for db(3)
Stores messages people sent to you
○ ~/.vacation.{dir,pag}
■ default database file for dbm(3)
○ ~/.[Link]
■ default message to send
● Use with forward(5)
○ \lctseng, |/usr/bin/vacation
33