MODULE 4

SYSTEM SECURITY

System Security: Introduction, Policy, Networks, Users, Authentication,

Processes, Files, Retrospective
(Text 3: Chapter 17)

Prepared By:
Dr. Shruthi M
Associate Professor
Department of Electronics and Communication

1
 1. Policy
Policy is at the heart of every decision involving security. The DMZ Web server
has a policy very different from that of the development system.
The Web Server System in the DMZ
The basic security policy of the Web server. Some of the consequences of the
policy are as follows.
1. All incoming Web connections come through the outer firewall, and all
replies are sent through the outer firewall.
2. All users log in from an internal trusted server running SSH. Web pages are
never updated locally. New Web pages are downloaded through the SSH tunnel.
3. Log messages are transmitted to the DMZ log server only.
4. The Web server may query the DMZ DNS system for IP addresses
5. Other than those expressly mentioned here, no network services are
provided.
6. The Web server runs CGI scripts. One of these scripts will write enciphered
information (transaction data) to a spooling area. The enciphered file will be
retrieved from the trusted internal administrative host using the SSH tunnel.
7. The Web server must implement its services correctly, and must restrict
access to those services as much as possible. 8. The public key of the principal
who will decipher and process the transaction data must reside on the DMZ
Web server.

The Web server consequences (WCs) of interest are as follows.

WC1. Policy consequence 1 requires that no unrequested network connections
except those from the outer firewall over the HTTP and HTTPS ports, and those
from the internal trusted administrative server over SSH, should be accepted.
Replies to DNS queries should be accepted provided that they come from the
DMZ DNS server. If other network clients are to be run, only replies to
messages originating from the DMZ Web server should be accepted.

2
WC2. Policy consequence 2 states that user access to the system is to be limited
to those users on the internal trusted administrative server. Furthermore, the
number of users who need access to the Web server should be as small as
possible, with only those privileges needed to perform their tasks. All actions
must be attributable to an individual, as opposed to a role, user.
WC3. Policy consequences 4 and 5 suggest that the Web server should be
configured to provide minimal access to the system. This prevents an attacker
who compromises the Web server from accessing other parts of the system. This
requirement leads to one unexpected, interesting consideration. If an attacker
gains access to the system through the Web server, she can delete all uncollected
transaction files. This denial of service attack would blemish the Drib’s
reputation. Some other mechanism should capture the transaction files and copy
them to an area that the Web server cannot reach. Then, if an attacker
compromises the Web server, that attacker cannot reach the transaction files.
WC4. Policy consequences 5, 6, and 8 imply that all software must have a very
high assurance of functioning correctly (as specified by its documentation). In
practice, this means that the software must be either developed or checked very
carefully. It also requires that extensive logging occur, to verify that the
software is functioning correctly even when under attack. In essence, we view
attacks as situations in which software functions correctly (and the attack fails)
or incorrectly (and the attack succeeds).
WC5. Policy consequence 7 states that the Web server must contain as few
programs, and as little software, configuration information, and other data, as
possible. If the system is compromised, this will minimize the effects of the
attack.

The Development System

The devnet has both infrastructure and user systems. The infrastructure sys tems
are the devnet firewall (which separates it from other internal subnets), a DNS
server, a logging host (which provides a central repository for logs), one or
more file servers, and one or more systems containing user information
common to the work stations (the UINFO servers). There is also an isolated
system used to build a “base system configuration” (system files, configuration
files, company-approved software, and so on) and to burn CD-ROMs.
The components of the security policy are as follows.

3
1. Only authorized users are allowed to use the devnet systems. They may work
on any devnet workstation. All actions and system accesses must be tied to an
individual user, rather than to a role account.
2. Workstation system administrators must be able to access the workstations at
all times, unless the particular workstation has crashed. The set of devnet
workstation administrators differs from the set of devnet central server
administrators.
3. Within the devnet itself, users are trusted not to attack devnet systems. Users
not on the devnet are not trusted. They are not allowed to access devnet
resources except as permitted by the network security policy (for internal Drib
users). Furthermore, devnet users are not allowed to access systems not on the
devnet except as permitted by the network policy.
4. All network communications, except electronic mail, are to be confidential
and are to be checked to ensure that the messages are not altered in transit.
5. The base standard configuration for each devnet system cannot be changed
on that system. There is to be a local area in each system in which developers
may install programs that are nonstandard. Before doing this, they must obtain
approval from the security officers and system administrators. Should the
software prove useful, it may be integrated into the standard configuration.
6. Backups shall enable system administrators to restore any devnet system
with the loss of at most one day’s changes in user and local files.
7. Security officers shall perform both periodic and ongoing audits of devnet
systems. Compromised systems shall be removed from the devnet until they
have been restored to an uncompromised state.
Other consequences of the policy apply to the devnet workstations. The
devel opment system consequences (DCs) of interest are as follows.
DC1. Policy components 1 and 4 imply the need for authenticated, enciphered,
integrity-checked communications. These policy components also imply a
consistent naming scheme across systems, so that a user name refers to the same
user on all devnet systems.
DC2. Policy component 2 requires that each workstation have one or more local
privileged accounts to administer the system locally. Policy components 1 and 2
imply that multiple local administrative accounts may be used to limit access to
particular administrative functions. This division of power into roles allows the
administrators to designate special system accounts, such as mail, as being

4
limited in their power. Policy requirement 2 also requires that the workstation
be able to run without any network connections.
DC3. Policy component 1 also requires that there be a notion of a “login” or
“audit” user. This identity must be recorded in logs, to tie individuals to actions.
Furthermore, users should not be able to log directly into role accounts such as
root, because this would eliminate the ability to tie an individual to an action.
Instead, they must log into an individual account and change to the role account,
or add a new role, to their individual account.
DC4. If a developer wants to install a program from the outside onto his devnet
workstation, he must first obtain approval from the security officers. Once
approved, he installs it in an area separate from the base system configuration
(see policy component 5). Adding a program to the base system configuration
requires that it be added to the isolated system first. This requires testing and
analysis of the program to ensure (to an appropriate level of integrity) that the
software is not malicious and will not accidentally damage the system on which
it runs.
DC5. Policy component 5 requires that each workstation protect the base
system configuration, as installed, from being altered. One approach is to mount
the disks containing that configuration as read-only disks. A far simpler and
more effective approach is to use read-only media. This meets policy
requirements and ensures that all devnet workstations are up to date. A writable
hard drive provides space for local files such as spool and temporary files.
DC6. Policy component 1 requires that an employee’s files be available to her
continuously. This requires that the files be stored on systems other than the
workstations, in case a workstation goes down. As a corollary, the file controls
should enforce the same sets of permissions regardless of the workstations from
which they are accessed.
DC7. Policy component 6 requires regular backups. As explained in Section
24.7.2, the development workstations store only transient files on writable
media. Hence, they need not be backed up. Restoration involves rebooting and
remounting of file systems from the file servers, which are regularly backed up.
DC8. Policy component 7 requires several security precautions. The primary
one is a logging system to which all systems send log messages. Furthermore,
security officers need access to both devnet systems and the devnet network.
They conduct periodic (and irregular) sweeps of the network, looking for
unauthorized servers. They also conduct periodic (and irregular) sweeps of each
system looking for dangerous settings in user accounts and the local areas.

5
 2. Networks
Both the DMZ Web server system and the devnet user system are
connected to the network. Although the firewalls provide some measure
of protection, the principle of separation of privilege says that access
should be limited even when the firewalls fail.3 So we consider how the
administrators should set network configurations and services to protect
the systems in the case that the firewalls fail.

The Web server is placed in the DMZ and must be accessed only through
controlled entry points.

1. Network Access Control (WC1)

 External users can reach the Web server only through the outer
firewall’s Web proxy.
 Internal administrators can access it only through SSH from a trusted
administrative system via the inner firewall.
 All other connection types or sources must be blocked.

2. Monitoring (WC4)
 The system must monitor all connection attempts to ensure the security
policy works and to detect failures.

3. Web Request Filtering

 The Web server accepts requests only from the outer firewall.
 It can also accept requests from the inner firewall if administrators
temporarily enable a Web proxy for debugging.
 It must reject all requests from other DMZ machines.

4. Apache Access Control Example

Using Apache configuration:
 order allow,deny → evaluates allow rules first.
 allow from outer_firewall
 allow from inner_firewall

6
  denyfromall
Only firewalls can send Web requests.

5. Administrative SSH Access

 The Web server runs an SSH server for secure administrative work.
 SSH requires authentication of both host and user.
 Only the trusted administrative server is allowed to connect.
Example:
 allow trusted_admin_server
 denyall
This ensures only authorized admins can log in.

6. High Availability
 Each service is wrapped with a script that automatically restarts it if it
stops, ensuring maximum uptime.

The Development System

[Link] Access (DC1)

 Developers can connect to the development system only through
encrypted and authenticated SSH.
 Both users and machines authenticate using public keys.

7
2. Services Running on the Development System
 It runs only the servers needed for development:
o Printer spooler
o Logging server
o Access servers for file and user database systems
 No FTP or Web servers run on development machines.
3. Centralized Services
 Special FTP and Web servers mount users’ directories from the central
file server.
 Workstations run a simple SMTP server but all mail is stored on a central
mail server.
 This setup:
o Reduces unnecessary services on each workstation.
o Limits the number of systems firewalls must allow.

[Link] Control Wrappers (DC2)

 Workstations use TCP wrappers to control access.
 These wrappers:
o Check where a network request comes from.
o Allow or block it based on rules.
o Log all connection attempts.
 Even if the firewall fails, the workstation still enforces security rules.
5. Logging (DC8)
 All access attempts are logged to detect intrusions and verify security.
6. Port Scanning by Security Officers
 Security teams regularly scan workstations:
o Check which ports are open.
o Compare with expected ports.
o Detect unauthorized systems or changes.
 Scans occur at random times to avoid predictability.
7. Penetration Testing
 Security officers also attack the system occasionally to test defenses.
 If vulnerabilities are found:
o Fixes are made in configuration.
o Or user-caused issues are investigated further.

8
3. Users

Our first step is to determine the accounts needed to run the systems. The
user accounts, as distinguished from the system administration accounts
(system administrators), require enough privileges to use the computer to
perform their jobs, but as few others as possible.

The Web Server System in the DMZ

To improve security, the number of user accounts on the Web server
should be minimal—typically just two users plus a sysadmin.
 User 1: Reads and serves web pages, writes to the web transaction area.
 User 2: Moves files from the web transaction area to the commerce
transaction area.
The Web server user must have very limited privileges because it is
exposed to the Internet and may be compromised. If it has only minimal
rights, an attacker cannot gain full system control.
The Web server user and commerce server user must be separate so
that a compromised Web server cannot delete or modify commerce data.
Access control helps, but security should never depend on just one
mechanism.

9
The Development System
User account per developer (items DC1, DC3, and DC6). It also requires
administrative accounts, as well as groups corresponding to projects (items DC2
and DC3). Furthermore, an account on different development systems must refer
to the same individual, role, or project (item DC1). Otherwise, inconsistent use
of identifiers may allow access rights that exceed the level authorized by the
security policy.

The development system is not accessible to users from the Internet. The outer
firewall, inner firewall, and devnet firewall prevent any direct connections from
the Internet. The threat comes from insiders, people with access to the Drib’s
internal network. The security analysts classified these threats into two distinct
sets.

10
 1. The first set involves nonadministrative information. This data is
sent enci phered and integrity-checked, using mechanisms that the
analysts trust. Compromising this data could lead to corruption of
user-specific data, and the analysts felt that the other mechanisms
provided sufficient protection to deter this.
2. The second set involves administrative information—specifically,
the NIS user records. These records are not encrypted. However,
none of these records include administrative accounts, so only
ordinary users can be compromised. The security analysts
configured each workstation so that only root could inject false
information that either the clients or the server would accept as
legitimate. They then physically secured the network to prevent
unauthorized personnel from connecting workstations to the Drib’s
network. Fake NIS replies can be put on the network only from the
outside (such replies would have to go through the devnet firewall)
or from a host on the network (such replies would require root
access). In the first case, the devnet firewall would reject the
packets before they entered the devnet network. In the second case,
root could access that user’s account by running the su command
on the system under attack, making unnecessary the injection of
false NIS packets to obtain access to a user’s account.

[Link]
Authentication binds the identity of the user to processes. Incorrect or
compromised authentication leads to security problems.
The Web Server System in the DMZ
 The SSH server uses cryptographic authentication so that only the trusted
admin machine can connect. If any other computer tries, the connection is
rejected. SSH does not depend on IP addresses; it uses strong
cryptography to verify identity.
 If cryptographic login fails, SSH asks for a password. Because the
organization is testing different smart card systems, the admin uses PAM
so they can change authentication methods easily without modifying SSH
source code.
 The Web server uses MD5-based password hashing, which allows long
and complex passwords. Users must choose passwords with letters,

11
 numbers, punctuation, and even spaces. The system checks if the
password is too easy before accepting it.
 Password aging is disabled because only trusted users can access the Web
server through the trusted admin host. Since no untrusted person can
attempt to guess passwords, forcing regular password changes is
unnecessary.

Development Network System

The development system supports several users. It is in a physically secure area,
accessible only to Drib employees. However, employees other than developers
(such as custodians and managers) have access to the restricted area, so
authentication con trols are required.
Each user must authenticate themselves when logging in (DC1).
Currently, users use reusable passwords. Since they are not administrators,
cracking their passwords could give attackers extra privileges. So password
aging is enabled.
Password rules on the development system:
 A password cannot be changed again for 3 days after it is set.
 Users must change their passwords every 90 days.
 They get a warning 1 week before expiration.
 After expiration, they must set a new password before fully logging in.
 Proposed passwords are checked for complexity using MD5-based
hashing and wordlist checks.
The system uses PAM to provide a simple and consistent authentication
interface.
The development system runs an SSH server that accepts connections from the
internal network. It authenticates:
 the host using public keys
 the user through public keys, smart cards, or passwords
To meet DC3, root login is disabled.
Admin must:
1. Log in as a normal user,

12
 2. Then switch to root using another program that verifies their identity and
authorization.
Role accounts cannot be directly logged into because their password hashes
cannot be matched by any real password.

4. Processes
A system runs a collection of processes to perform specific tasks. Each process
is a potential vulnerability. This section examines the processes run on both
systems.
The Web Server System in the DMZ
The required services are as follows.
o Web server
o Commerce server
o SSH server
o Login server, if there is a physical terminal or console
o Any essential operating system services (such as pagers)

13
14
 5. Files

The setting of protection modes, and the contents of files, affect the
protection domains of users and so are critical to a system satisfying a
security policy. Again, consider each system separately.

The Web Server System in the DMZ

The goal of the Web server system is to serve web pages safely.
To protect the system (as required by WC4), all important system files and
programs are stored on a read-only CD-ROM, so attackers cannot change
them even if they break into the server.
How the file system is arranged:
 The system boots from a CD-ROM, which cannot be modified.
 CGI programs, configuration files, and the public key are also stored on
the CD-ROM for protection.
 Web pages (which change often) are stored on the hard drive, mounted
inside the web directory. Attackers can change web pages, but cannot
modify CGI programs or system files.
At startup, two directories from the hard drive are mounted:
1. The web pages directory
2. A user area containing admin home directories, temporary files, and
transaction spools
The Web server is confined (WC3) to the web directory, meaning it can only
read or write inside that part of the system.
So even if attackers take over the Web server, they cannot:
 Change system files
 Modify CGI programs
 Access the commerce server
Security of Electronic Transactions:
 CGI programs encrypt all transaction data using a public key before
saving it.
 The corresponding private key is kept on an internal system, not on the
DMZ server.
 So attackers cannot read or tamper with the sensitive data.
Example Scenario (simplified):
 Web server runs as user webbie
 Commerce server runs as ecommie
 Only the "pages" directory on the hard drive is writable by both
 CGI programs and keys are owned by root and stored on the CD-ROM

15
  Commerce server moves each completed transaction to its own secure
spool area
Minimizing Programs (WC5):
The system includes only essential programs (login shell, file utilities,
server control).
No compilers, browsers, cron, mail, or development tools.
This limits what an attacker can use if they break in.
Integrity Checking (WC4):
Periodically, the system is:
1. Stopped
2. Booted from CD-ROM
3. Hard drive reformatted
4. Web pages and user data restored from a known clean internal copy
This ensures the system stays in a safe, trusted state and eliminates any hidden
backdoors.

The Development System

The development system’s goal is to provide the resources that developers need
to develop new software for the Drib’s products and (if necessary)
infrastructure and systems. This requires a variety of software. A site can take
two approaches.
 The first approach is to allow each developer to configure his or her own
work station. The Drib rejected this approach because it would create too
many different systems for the system administrators to manage.
Furthermore, tools available on one workstation might not be available on
another, violating the interchangeability required by item DC6.
 The second approach is to develop a standard configuration that provides
developers and system administrators with needed software tools and
configuration settings. To create such a configuration, the Drib policy
managers gathered developers, system administrators, security officers,
and all other users of the development workstations. The group developed
a configuration that met the Drib’s policies and that was acceptable to as
many people as possible, and ensured that all members of the group were
willing and able to use systems with that configuration.75 The system
administrators then installed and configured a base system on an isolated
workstation system and created a bootable CD-ROM. This CD-ROM was
copied and given to all developers.

16
EXAMPLE: The UNIX file scanning program binaudit [98, 103] allows the
system administrator to describe the names and attributes of files. If files do not
match the attributes, an alarm is raised. Other programs such as Tiger [768],
COPS [309], and TITAN [308] report files with names that match suspicious
patterns. The Drib’s developers developed their own tool to perform both of
these functions.

5. Retrospective

The Web Server System in the DMZ

 The DMZ Web server runs minimal services and keeps system files on
read-only media to prevent tampering.
 Only encrypted, authenticated connections from a trusted internal host
are allowed, except for the public web requests.
 Public web requests pass through an outer firewall, which can block
attacks such as denial-of-service.
 The Web server and commerce server run with minimal privileges and
communicate only through an encrypted shared directory for transaction
files.
 Transaction files are public-key encrypted, so attackers can only delete,
not modify them.
 The commerce server quickly moves files to a location inaccessible to the
Web server.
 Administrative access requires going through a trusted internal host with
public key authentication.
 The SSH server rejects all other hosts, verifying identity by keys, not IP
addresses.
 All unnecessary programs/services are deleted, reducing maintenance and
preventing attacker misuse.

The Development System

 The development system also runs a minimal set of programs, but

“minimal” differs from the DMZ server because developers need tools for
compiling, debugging, testing, and integrating software.
 Security is important but not the main focus; the system is protected by
three firewalls and relies on the overall infrastructure for safety.
17
 Many users can access the system from development network machines and
possibly other internal subnets.
 User info is stored in a centralized repository; reusable passwords are
allowed, and password aging is not enforced, but passwords must be strong and
are periodically checked by security officers.
 Alternative authentication schemes are supported.
 Daily backups are performed. Each workstation has a local writable area
that is also backed up, mainly to preserve log files for potential investigations.

18