1

CSC314: OS Security
Week 10
Chapter 14 and 15 in Operating system concepts 9th edition (Silberschatz)

Dr Nyamsi 5/5/2022
2
Outline

 Protection

Protection definition

Protection mechanisms

 Security

Security issues

Some cryptographic themes

Dr Nyamsi 5/5/2022
3
Concepts
 Protection definition:
 Mechanisms and policy to keep programs and users from
accessing or changing stuff they should not do

 Is Internal to OS

 Security:
 Issues are external to OS

 Authentication of user, validation of messages, malicious or

accidental introduction of flaws, etc.

Dr Nyamsi 5/5/2022
4
Goal of protection

 In one protection model, computer consists of a collection of

objects (hardware or software)

 Each object has a unique name and can be accessed through

a well-defined set of operations (read, write, print, … etc.)

 Protection problem is to ensure that each object is accessed

correctly and only by those processes that are allowed to do so

Dr Nyamsi 5/5/2022
5
Protection mechanism: guideline
 Guiding principle – principle of least privilege

 Programs, users and systems should be given just enough privileges to

perform their tasks

 Limits damage if entity has a bug, gets abused

 Can be static (during life of system, during life of process)

 Or dynamic (changed by process as needed) – domain switching,

privilege escalation

 “Need to know” a similar concept regarding access to data

Dr Nyamsi 5/5/2022
6
Protection mechanism
 Principle of least privilege

 Programs, users and systems should be given just enough privileges to

perform their tasks

 Limits damage if entity has a bug, gets abused

 Can be static (during life of system, during life of process)

 Or dynamic (changed by process as needed) – domain switching,

privilege escalation

 “Need to know” a similar concept regarding access to data

Dr Nyamsi 5/5/2022
7
Protection mechanism

 Must consider “grain” aspect

Rough-grained privilege management easier, simpler, but

least privilege now done in large chunks

Fine-grained management more complex, more overhead,

but more protective

 Domain can be user, process, procedure

Dr Nyamsi 5/5/2022
8
Protection mechanism

 Separate policy from mechanism

Mechanism: the stuff built into the OS to make protection

work

Policy: the data that says who can do what to whom

Dr Nyamsi 5/5/2022
 Protection mechanism: domain structure
9

 Access-right = <object-name, rights-set>

where rights-set is a subset of all valid operations that can be
performed on the object.

 Domain = set of access-rights

Dr Nyamsi 5/5/2022
 Protection mechanism: domain structure
10

 Domain = user-id

 Domain switch accomplished via file system

 Each file has associated with it a domain bit (setuid bit)

 When file is executed and setuid = on, then user-id is set to owner of
the file being executed

 When execution completes user-id is reset

Dr Nyamsi 5/5/2022
 Protection mechanism: domain structure
11

 Domain switch accomplished via passwords

 Su (Switch User) command temporarily switches to another user’s

domain when other domain’s password provided

 Domain switching via commands

 Sudo (Super User Do) command prefix executes specified

command in another domain (if original domain has privilege or
password given)

Dr Nyamsi 5/5/2022
12
Protection mechanism

 View protection as a matrix (access matrix)

 Rows represent domains

 Columns represent objects

 Access(i, j) is set of operations that process executing in Domain

(Di) can invoke on Object (Oj)

Dr Nyamsi 5/5/2022
13
Protection mechanism
 Columns are access control lists (ACLs): Associated with each object

 Rows are capabilities: Associated with each user, group, or domain

Dr Nyamsi 5/5/2022
 Protection mechanism: use of access matrix
14

 If a process in Domain Di tries to do “op” on object Oj, then “op”

must be in the access matrix

 User who creates object can define access column for that object

 Access matrix can be expanded to dynamic protection

 Operations to add, delete access rights

Dr Nyamsi 5/11/2022
 Protection mechanism: use of access matrix
15

 Access matrix can be expanded to dynamic protection

 Special access rights:

owner of Oi

copy op from Oi to Oj (denoted by “*”)

control – Di can modify Dj access rights

transfer – switch from domain Di to Dj

 Copy and Owner applicable to an object

 Control applicable to domain object

Dr Nyamsi 5/11/2022
 Protection mechanism: use of access matrix
16

Access matrix design separates mechanism from policy

 Mechanism (an established process by which something takes place or is

brought about)
 Operating system provides access-matrix + rules

 If ensures that the matrix is only manipulated by authorized agents and that rules
are strictly enforced

 Policy (prudent or expedient conduct or action)

 User dictates policy

 Who can access what object and in what mode

 But doesn’t solve the general confinement problem

Dr Nyamsi 5/11/2022
 Protection mechanism: use of access matrix
17

Access Matrix with Domains as Objects

Dr Nyamsi 5/11/2022
 Protection mechanism: use of access matrix
18

Dr Nyamsi
Access Matrix with Copy Rights 5/11/2022
 Protection mechanism: use of access matrix
19

Dr Nyamsi
Access Matrix with Owner Rights 5/11/2022
20
Access matrix: Implementation
 Access matrix is generally, a sparse matrix
 Option 1 – Global table
Store ordered triples <domain, object, rights-set> in table
A requested operation M on object Oj within domain Di imply
search table for < Di, Oj, Rk > ; with M ∈ Rk
But table could be large, consequence: it won’t fit in main
memory
Difficult to group objects (consider an object that all domains
can read)

Dr Nyamsi 5/11/2022
21
Access matrix: Implementation
 Access matrix is generally, a sparse matrix
 Option 2 – Access lists for objects
Each column implemented as an access list for one object
Resulting per-object list consists of ordered pairs <domain,
rights-set> defining all domains with non-empty set of access
rights for the object.
Easily extended to contain default set
If M ∈ default set, also allow access

Dr Nyamsi 5/11/2022
22
Access matrix: Implementation
 Each column = Access-control list for one object
Defines who can perform what operation
Domain 1 = Read, Write
Domain 2 = Read
Domain 3 = Read

 Each Row = Capability List (like a key)

For each domain, what operations allowed on what objects
Object F1 – Read
Object F4 – Read, Write, Execute
Object F5 – Read, Write, Delete, Copy

Dr Nyamsi 5/11/2022
23
Access matrix: Implementation
 Option 3 – Capability list for domains
 Instead of object-based, list is domain based
 Capability list for domain is list of objects together with operations allows
on them
 Object represented by its name or address, called a capability
 Execute operation M on object Oj, process requests operation and
specifies capability as parameter
Possession of capability means access is allowed
 Capability list associated with domain but never directly accessible by
domain
Rather, protected object, maintained by OS and accessed indirectly
Like a “secure pointer”
Idea can be extended up to applications
Dr Nyamsi 5/11/2022
24
Access matrix: Implementation
 Option 4 – Lock-key

Compromise between access lists and capability lists

Each object has list of unique bit patterns, called locks

Each domain as list of unique bit patterns called keys

Process in a domain can only access object if domain has key

that matches one of the locks

Dr Nyamsi 5/11/2022
25
Access matrix: Implementation
 Comparison of implementations:

Global table is simple, but can be large

Access lists correspond to needs of users

Determining set of access rights for domain non-localized

so difficult

Every access to an object must be checked

Many objects and access rights -> slow

Dr Nyamsi 5/11/2022
26
Access matrix: Implementation
 Comparison of implementations:

Capability lists useful for localizing information for a given

process

But revocation capabilities can be inefficient

Lock-key effective and flexible, keys can be passed freely

from domain to domain, easy revocation

Dr Nyamsi 5/11/2022
27
Access matrix: Implementation
 Most systems use combination of access lists and capabilities

 First access to an object -> access list searched

If allowed, capability created and attached to process

Additional accesses need not be checked

After last access, capability destroyed

Consider file system with ACLs per file

Dr Nyamsi 5/11/2022
28
Access control
 To access a file, a resource, you need a privilege.

 Privilege is right to execute system call or use an option within a

system call

 Privilege can be assigned to processes

 Users assigned roles granting access to privileges and programs

Enable role via password to gain its privileges

 Similar to access matrix

Dr Nyamsi 5/11/2022
29
Access control
 To access a file, a resource, you need a privilege.

Dr Nyamsi 5/11/2022
30
Revocation of Access Rights
 As you grant access, you can do remove the access right to a file or resource

 Various options to remove the access right of a domain to an object

 Immediate vs. delayed

 Selective vs. general

 Partial vs. total

 Temporary vs. permanent

 Access List – Delete access rights from access list

 Simple – search access list and remove entry

 Immediate, general or selective, total or partial, permanent or temporary

Dr Nyamsi 5/11/2022
31
Revocation of Access Rights
 Capability List – Scheme required to locate capability in the system before
capability can be revoked
 Reacquisition – periodic delete, with require and denial if revoked

 Back-pointers – set of pointers from each object to all capabilities of that object
(Multics)

 Indirection – capability points to global table entry which points to object – delete entry
from global table, not selective (CAL)

 Keys – unique bits associated with capability, generated when capability created

 Master key associated with object, key matches master key for access

 Revocation – create new master key

 Policy decision of who can create and modify keys – object owner or others?

Dr Nyamsi 5/11/2022
32
Language-Based Protection
 Specification of protection in a programming language allows the
high-level description of policies for the allocation and use of resources

 Language implementation can provide software for protection

enforcement when automatic hardware-supported checking is
unavailable

 Interpret protection specifications to generate calls on whatever

protection system is provided by the hardware and the operating
system

Dr Nyamsi 5/11/2022
33
Language-Based Protection
 With Java for example,
 Protection is handled by the Java Virtual Machine (JVM)

 A class is assigned a protection domain when it is loaded by the JVM

 The protection domain indicates what operations the class can (and cannot)
perform

 If a library method is invoked that performs a privileged operation, the stack is

inspected to ensure the operation can be performed by the library

 Generally, Java’s load-time and run-time checks enforce type safety

 Classes effectively encapsulate and protect data and methods from other
classes

Dr Nyamsi 5/11/2022
34
Outline

 Protection

Protection definition

Protection mechanisms

 Security

Security issues

Some cryptographic themes

Dr Nyamsi 5/11/2022
35
Security
 The security problem
 System is secured if resources used and accessed as intended
under all circumstances

 Intruders (crackers) attempt to breach security

 Threat is potential security violation

 Attack is attempt to breach security

 Attack can be accidental or malicious

 Easier to protect against accidental than malicious misuse

Dr Nyamsi 5/11/2022
36
Security: Violation Categories

 Breach of confidentiality: Unauthorized reading of data

 Breach of integrity: Unauthorized modification of data

 Breach of availability: Unauthorized destruction of data

 Theft of service: Unauthorized use of resources

 Denial of service (DOS): Prevention of legitimate use

Dr Nyamsi 5/11/2022
37
Security: Violation Methods

 Masquerading (breach authentication): Pretending to be

an authorized user to escalate privileges

 Replay attack: As is or with message modification

 Man-in-the-middle attack: Intruder sits in data flow,

masquerading as sender to receiver and vice versa

 Session hijacking: Intercept an already-established session

to bypass authentication
Dr Nyamsi 5/11/2022
38
Security: Measure levels

 Impossible to have absolute security, but make cost to

perpetrator sufficiently high to deter most intruders

 Security must occur at four levels to be effective:

Physical: Data centers, servers, connected terminals

Human: Avoid social engineering, phishing, dumpster diving

Operating System: Protection mechanisms, debugging

Network: Intercepted communications, interruption, DOS

Dr Nyamsi 5/11/2022
39
Security: Program Threats

 Many variations, many names

 Trojan Horse
Code segment that misuses its environment

Exploits mechanisms for allowing programs written by users to

be executed by other users

Spyware, pop-up browser windows, covert channels

Up to 80% of spam delivered by spyware-infected systems

Dr Nyamsi 5/11/2022
40
Security: Program Threats

 Trap Door

Specific user identifier or password that circumvents normal

security procedures

Could be included in a compiler

 Logic Bomb

Program that initiates a security incident under certain

circumstances

Dr Nyamsi 5/11/2022
41
Security: Program Threats

 Stack and Buffer Overflow

 Exploits a bug in a program (overflow the stack or memory buffers)

 Failure to check bounds on inputs, arguments

 Write past arguments on the stack into the return address on stack

 When routine returns from call, returns to hacked address

Pointed to code loaded onto stack that executes malicious code

 Unauthorized user or privilege escalation

Dr Nyamsi 5/11/2022
42
Security: Program Threats

 Viruses

Code fragment embedded in legitimate program

Self-replicating, designed to infect other computers

Very specific to CPU architecture, operating system,

applications

Usually borne via email or as a macro

Dr Nyamsi 5/11/2022
43
Security: Program Threats

 Virus dropper inserts virus onto the system

 Many categories of viruses, literally many thousands of viruses

 File / parasitic

 Boot / memory

 Macro

 Source code

 … etc.

Dr Nyamsi 5/11/2022
44
Security: Systems and network Threats

 Some systems “open” rather than secure by default

Reduce attack surface

But harder to use, more knowledge needed to administer

 Network threats harder to detect, prevent

Protection systems weaker

More difficult to have a shared secret on which to base

access

Dr Nyamsi 5/11/2022
45
Security: Systems and network Threats

 Network threats harder to detect, prevent

No physical limits once system attached to internet

Or on network with system attached to internet

Even determining location of connecting system difficult

IP address is only knowledge

Dr Nyamsi 5/11/2022
46
Security: Systems and network Threats

 Port scanning

Automated attempt to connect to a range of ports on one or

a range of IP addresses

Detection of answering service protocol

Detection of OS and version running on system

Dr Nyamsi 5/11/2022
47
Security: Systems and network Threats
 Port scanning

 Nmap (Network Mapper) is a free and open-source utility for

network discovery and security auditing. It scans all ports in a given
IP range for a response

 Nessus (Security Scanning Tools Worldwide) has a database of

protocols and bugs (and exploits) to apply against a system

 Frequently launched from zombie systems

To decrease trace-ability

Dr Nyamsi 5/11/2022
48
Security: Systems and network Threats
 Denial of Service

 Overload the targeted computer preventing it from doing any useful

work

 Distributed denial-of-service (DDOS) come from multiple sites at once

 Consider the start of the IP-connection handshake (SYN)

How many started-connections can the OS handle?

Dr Nyamsi 5/11/2022
49
Security: Systems and network Threats

 Denial of Service

 Consider traffic to a web site

How can you tell the difference between being a target and being
popular?

 Accidental – students writing bad fork() code

 Purposeful – extortion, punishment

Dr Nyamsi 5/11/2022
 Security: Cryptography as a Security Tool
50

 Broadest security tool available

 Internal to a given computer, source and destination of messages can

be known and protected

 OS creates, manages, protects process IDs, communication ports

 Source and destination of messages on network cannot be trusted

without cryptography

 Local network – IP address? Consider unauthorized host added

 WAN / Internet – how to establish authenticity ? Not via IP address

Dr Nyamsi 5/11/2022
 Security: Cryptography as a Security Tool
51

 Cryptography is a means to constrain potential senders (sources)

and/or receivers (destinations) of messages

 Based on secrets (keys)

 Enables

Confirmation of source

Receipt only by certain destination

Trust relationship between sender and receiver

Dr Nyamsi 5/11/2022
52
Security: Encryption
 Constrains the set of possible receivers of a message

 Encryption algorithm consists of

 Set K of keys

 Set M of Messages

 Set C of ciphertexts (=encrypted messages)

Dr Nyamsi 5/11/2022
53
Security: Encryption
 Encryption algorithm consists of

 A function E : K → (M→C). That is, for each k  K, Ek is a function for

generating ciphertexts from messages

 Both E and Ek for any k should be efficiently computable functions

 A function D : K → (C → M). That is, for each k  K, Dk is a function

for generating messages from ciphertexts

 Both D and Dk for any k should be efficiently computable functions

Dr Nyamsi 5/11/2022
54
Security: Encryption
 An encryption algorithm must provide this essential property: Given a
ciphertext c  C, a computer can compute m such that Ek(m) = c
only if it possesses k

 Thus, a computer holding k can decrypt ciphertexts to the

plaintexts used to produce them, but a computer not holding k
cannot decrypt ciphertexts

 Since ciphertexts are generally exposed (for example, sent on the

network), it is important that it be infeasible to derive k from the
ciphertexts
Dr Nyamsi 5/11/2022
55
Security: Symmetric Encryption
 Same key used to encrypt and decrypt

 Therefore, k must be kept secret

 DES (Data Encryption Standard) was most used symmetric block-

encryption algorithm (created by US Govt)

 Encrypts a block of data at a time

 Keys too short so now considered insecure

Dr Nyamsi 5/11/2022
56
Security: Symmetric Encryption
 2001 NIST adopted new block cipher - Advanced Encryption Standard
(AES)

 Keys of 128, 192, or 256 bits, works on 128 bit blocks

 RC4 is most common symmetric stream cipher, but known to have

vulnerabilities

 Encrypts/decrypts a stream of bytes (i.e., wireless transmission)

 Key is an input to pseudo-random-bit generator

 Generates an infinite keystream

Dr Nyamsi 5/11/2022
57
Security: Symmetric Encryption

Dr Nyamsi 5/11/2022
58
Security: Asymmetric Encryption
 Public-key encryption based on each user having two keys:
 public key – published key used to encrypt data
 private key – key known only to individual user used to decrypt
data
 Must be an encryption scheme that can be made public
without making it easy to figure out the decryption scheme
 Most common is RSA (Rivest–Shamir–Adleman) block cipher
 Efficient algorithm for testing whether a number is prime or not
 No efficient algorithm is known for finding the prime factors of a
number

Dr Nyamsi 5/12/2022
59
Security: Asymmetric Encryption
 Formally, it is computationally infeasible to derive kd,N from ke,N,
and so ke need not be kept secret and can be widely
disseminated
 ke is the public key
 kd is the private key
 N is the product of two large, randomly chosen prime numbers p
and q (for example, p and q are 512 bits each)
 Encryption algorithm is Eke,N(m) = mke mod N, where ke satisfies
kekd mod (p−1)(q −1) = 1
 The decryption algorithm is then Dkd,N(c) = ckd mod N

Dr Nyamsi 5/11/2022
60
Security: Asymmetric Encryption
 For example. Make p = 7 and q = 13
 We then calculate N = 7∗13 = 91 and (p−1)(q−1) = 72

 We next select ke relatively prime to 72 and < 72, yielding 5

 Finally, we calculate kd such that kekd mod 72 = 1, yielding 29

 We how have our keys

 Public key, ke,N = 5, 91

 Private key, kd,N = 29, 91

Dr Nyamsi 5/11/2022
61
Security: Asymmetric Encryption
 For example. Make p = 7 and q = 13
Encrypting the message 69 with the public key results in the
cyphertext 62

Cyphertext can be decoded with the private key

Public key can be distributed in cleartext to anyone who

wants to communicate with holder of public key

Dr Nyamsi 5/11/2022
 Security: Asymmetric Encryption
62

Dr Nyamsi 5/11/2022
 Security: Asymmetric Encryption
63

 Note

Symmetric cryptography based on transformations,

Asymmetric based on mathematical functions

Asymmetric much more compute intensive

Typically, not used for bulk data encryption

Dr Nyamsi 5/11/2022
 Authentication
64

 Constraining set of potential senders of a message

 Complementary to encryption

 Also can prove message unmodified

 Algorithm components

 A set K of keys

 A set M of messages

 A set A of authenticators

Dr Nyamsi 5/12/2022
 Authentication
65
 Algorithm components
 A function S : K → (M→ A)

That is, for each k  K, Sk is a function for generating authenticators from

messages

Both S and Sk for any k should be efficiently computable functions

 A function V : K → (M × A→ {true, false}).

That is, for each k  K, Vk is a function for verifying authenticators on

messages

Both V and Vk for any k should be efficiently computable functions

Dr Nyamsi 5/12/2022
 Authentication
66

 For a message m, a computer can generate an authenticator a

 A such that Vk(m, a) = true only if it possesses k

 Thus, computer holding k can generate authenticators on

messages so that any other computer possessing k can verify
them

 Computer not holding k cannot generate authenticators on

messages that can be verified using Vk

Dr Nyamsi 5/12/2022
 Authentication
67

 Since authenticators are generally exposed (for example, they

are sent on the network with the messages themselves), it must
not be feasible to derive k from the authenticators

 Practically, if Vk(m,a) = true then we know m has not been

modified and that send of message has k

 If we share k with only one entity, know where the message

originated

Dr Nyamsi 5/12/2022
 Authentication – Hash function
68

 Hash function is the basis of authentication

 Creates small, fixed-size block of data message digest (hash

value) from m

 Hash Function H must be collision resistant on m

 It must be infeasible to find an m’ ≠ m such that H(m) = H(m’)

 If H(m) = H(m’), then m = m’

 The message has not been modified

Dr Nyamsi 5/12/2022
 Authentication – Hash function
69

 Common message-digest functions include MD5, which

produces a 128-bit hash, and SHA-1, which outputs a 160-bit
hash

 Not useful as authenticators

 For example, H(m) can be sent with a message

 But if H is known someone could modify m to m’ and recompute

H(m’) and modification not detected

 So must authenticate H(m)

Dr Nyamsi 5/12/2022
 Authentication – Digital Signature
70

 Based on asymmetric keys and digital signature algorithm

 Authenticators produced are digital signatures

 Very useful – anyone can verify authenticity of a message

 In a digital-signature algorithm, computationally infeasible to

derive ks from kv

 V is a one-way function

 Thus, kv is the public key and ks is the private key

Dr Nyamsi 5/12/2022
 Authentication – Digital Signature
71

 Consider the RSA digital-signature algorithm

 Like the RSA encryption algorithm, but the key use is reversed

 Digital signature of message Sks (m) = H(m)ks mod N

 The key ks again is a pair (d, N), where N is the product of two
large, randomly chosen prime numbers p and q

 Verification algorithm is Vkv(m, a) = (akv mod N = H(m))

Where kv satisfies kvks mod (p − 1)(q − 1) = 1

Dr Nyamsi 5/12/2022
 Authentication – Digital Signature
72

 Why authentication if a subset of encryption?

 Fewer computations (except for RSA digital signatures)

 Authenticator usually shorter than message

 Sometimes want authentication but not confidentiality

 It can be basis for non-repudiation

Dr Nyamsi 5/12/2022
 Security : Key distribution
73

 Delivery of symmetric key is huge challenge

 Sometimes done out-of-band

 Asymmetric keys can proliferate – stored on key ring

 Even asymmetric key distribution needs care (man-in-the-middle

attack)

Dr Nyamsi 5/12/2022
 Security : key and digital certification
74

 Proof of who or what owns a public key

 Public key digitally signed a trusted party

 Trusted party receives proof of identification from entity and certifies

that public key belongs to entity

 Certificate authority are trusted party – their public keys included with
web browser distributions

 They vouch for other authorities via digitally signing their keys, and so on

Dr Nyamsi 5/12/2022
75
Security: Example of Encryption
 Insertion of cryptography at one layer of the ISO network model (the
transport layer)
 SSL – Secure Socket Layer (also called TLS)
 Cryptographic protocol that limits two computers to only exchange
messages with each other
 Very complicated, with many variations
 Used between web servers and browsers for secure communication
(credit card numbers for example)
 The server is verified with a certificate assuring client is talking to correct
server
 Asymmetric cryptography used to establish a secure session key
(symmetric encryption) for bulk of communication during session
 Communication between each computer then uses symmetric key
cryptography
Dr Nyamsi 5/11/2022
 Security: Example of user authentication
76

 Crucial to identify user correctly, as protection systems depend

on user ID
 User identity most often established through passwords, can be
considered a special case of either keys or capabilities
 Passwords must be kept secret
 Frequent change of passwords
 History to avoid repeats
 Use of “non-guessable” passwords
 Log all invalid access attempts (but not the passwords
themselves)
 Unauthorized transfer
Dr Nyamsi 5/12/2022
 Security: Example of user authentication
77

 Passwords may also either be encrypted or allowed to be used

only once
 Does encrypting passwords solve the exposure problem?
 Might solve sniffing
 Consider shoulder surfing
 Consider Trojan horse keystroke logger
 Question:
 How are passwords stored at authenticating site?

Dr Nyamsi 5/12/2022
78
Security: Password
 Encrypt to avoid having to keep secret
 But keep secret anyway (i.e. Unix uses superuser-only readably file
/etc/shadow)
 Use algorithm easy to compute but difficult to invert
 Only encrypted password stored, never decrypted
 Add “salt” to avoid the same password being encrypted to the
same value
 One-time passwords
 Use a function based on a seed to compute a password, both
user and computer
 Hardware device / calculator / key fob to generate the password
Dr Nyamsi 5/12/2022
79
Security: Password
 Biometrics

 Some physical attribute (fingerprint, hand scan)

 Multi-factor authentication

 Need two or more factors for authentication

 i.e., USB ”dongle”, biometric measure, and password

Dr Nyamsi 5/12/2022
80
Security: Password

Questions ?

Dr Nyamsi 5/12/2022