Operating Systems

Unix / Linux – Complete Study Notes

Comprehensive Reference for Exams & Practicals

✦ Introduction to OS & OS Structures

✦ Process Management & CPU Scheduling
✦ Thread Management & Concurrency / Synchronization
✦ Deadlocks & Memory Management
✦ Virtual Memory & File Systems
✦ Secondary Storage & I/O Systems
✦ Protection & Security
✦ Linux: Process Mgmt, File Permissions, Startup/Shutdown
✦ Unix: Commands, File System, Permissions, Shell Programming
✦ Filtering, Pattern Matching & Text Processing Commands

Includes command tables, algorithms, examples and system call references

OS & Unix/Linux Notes Page 1

 CHAPTER 1: Introduction to Operating Systems
1.1 What is an Operating System?
An Operating System (OS) is system software that acts as an intermediary between computer hardware and
application software. It manages hardware resources and provides services for application programs. Without
an OS, every application would need to directly manage CPU, memory, and I/O — impractical and unsafe.

Primary Goals of an OS:

• Convenience: Makes the computer easy to use (abstracts hardware complexity).

• Efficiency: Manages resources (CPU, memory, I/O) for maximum utilisation.
• Ability to Evolve: Permits development and testing of new functions without disrupting services.

1.2 OS Roles
Resource Manager Allocates CPU time, memory space, file storage, I/O devices among competing
processes.

Extended Machine Provides a virtual machine abstraction — programmers write to OS API, not bare
hardware.

Kernel The core of the OS; always resident in memory; handles most critical operations.

Shell User interface to the kernel — CLI (bash, zsh) or GUI (Windows Explorer,
GNOME).

1.3 Types of Operating Systems

Batch OS Jobs collected, grouped, and processed without user interaction. Early
mainframes.

Multiprogramming OS Multiple jobs in memory; CPU switches when I/O occurs. Maximises CPU
utilisation.

Time-Sharing OS CPU switches rapidly among users (multitasking). Each user gets time slice.
UNIX, Linux.

Real-Time OS (RTOS) Hard deadlines for task completion. Hard RTOS: miss = failure (pacemakers).
Soft RTOS: delay tolerated (streaming).

Distributed OS Manages a group of networked computers as a single coherent system.

Embedded OS Designed for specific hardware (phones, routers). Minimal footprint. FreeRTOS,
VxWorks.

Mobile OS Optimised for mobile hardware constraints. Android (Linux kernel), iOS (XNU
kernel).

Network OS Provides services to network nodes. NFS, file sharing, remote login.

1.4 OS Services
• Program Execution: Load program into memory, run it, handle errors.

OS & Unix/Linux Notes Page 2

 • I/O Operations: Manage I/O devices; applications use OS calls, not hardware directly.
• File System Manipulation: Create, read, write, delete files and directories.
• Communications: Inter-process communication (IPC) via shared memory or message passing.
• Error Detection: Detect and correct errors in CPU, memory, I/O devices, user programs.
• Resource Allocation: Allocate CPU cycles, memory, I/O ports to multiple users/jobs.
• Accounting: Track resource usage for billing or statistics.
• Protection & Security: Control access to resources; authenticate users.

1.5 System Calls

A system call is a programmatic interface between a running process and the OS kernel. Applications run in
user mode; system calls switch to kernel mode to perform privileged operations. This protection boundary
prevents user code from corrupting the OS.

Process Control fork(), exec(), exit(), wait(), kill()

File Management open(), read(), write(), close(), lseek(), stat()

Device Management ioctl(), read(), write() on device files

Information getpid(), alarm(), sleep(), time()

Communication pipe(), shmget(), mmap(), socket(), send(), recv()

Protection chmod(), chown(), setuid(), getuid()

■ Note: In Linux, system calls are invoked via the int 0x80 instruction (32-bit) or syscall instruction (64-bit). The C
library (glibc) wraps them in convenient functions.

OS & Unix/Linux Notes Page 3

 CHAPTER 2: Operating System Structures
2.1 OS Architecture Models

2.1.1 Monolithic Kernel

• Entire OS runs in kernel space as a single large process.
• All OS services (file system, drivers, scheduling, memory) in one address space.
• Fast — no context switches between kernel components.
• Dangerous — a bug in any module can crash the entire system.
• Examples: Traditional Unix, Linux (modular monolithic — loadable kernel modules).

2.1.2 Layered Approach

• OS divided into layers. Layer 0 = hardware; Layer N = user interface.
• Each layer uses services only from layers below it.
• Easy to debug and verify, but poor performance (multiple layer traversals).
• Difficult to define clean layer boundaries.

2.1.3 Microkernel
• Kernel contains only essential services: IPC, basic scheduling, memory management.
• Other OS services (file systems, drivers, networking) run as user-space servers.
• More reliable — server crash doesn't bring down kernel.
• Slower — frequent message passing between user-space servers.
• Examples: Mach, QNX, L4, MINIX 3, macOS (hybrid — Darwin/XNU has microkernel core).

2.1.4 Hybrid Kernel

• Combines monolithic speed with microkernel modularity.
• Performance-critical components in kernel space; others in user space.
• Windows NT, macOS XNU, DragonFly BSD.

2.1.5 Modular Kernel (Linux approach)

• Monolithic base with support for loadable kernel modules (LKMs).
• Modules can be loaded/unloaded at runtime: insmod, rmmod, lsmod.
• Device drivers, file systems, network protocols as modules.
• Best of both worlds: performance + flexibility.

2.2 Virtual Machines

A Virtual Machine (VM) creates an illusion of multiple operating systems running on one physical machine. The
hypervisor (VMM — Virtual Machine Monitor) manages this.

Type 1 Hypervisor Runs directly on hardware (bare-metal). Examples: VMware ESXi, Xen, KVM,
Hyper-V.

Type 2 Hypervisor Runs on top of a host OS. Examples: VirtualBox, VMware Workstation, QEMU.

Containerisation OS-level virtualisation. Shares host kernel; isolated user spaces. Docker, LXC.
Lighter than VMs.

OS & Unix/Linux Notes Page 4

 JVM Java Virtual Machine — language-level VM providing platform independence.

2.3 Boot Process

• Power On: CPU executes BIOS/UEFI firmware from ROM.
• POST: Power-On Self Test — checks hardware integrity.
• BIOS/UEFI: Finds bootable device; loads MBR/GPT boot sector.
• Bootloader (GRUB2): Loads OS kernel into memory, sets up initial environment.
• Kernel Init: Kernel decompresses, initialises subsystems (memory, drivers, VFS).
• Init/Systemd: First user-space process (PID 1); starts all other services.
• Login: Display manager or getty presents login prompt.

OS & Unix/Linux Notes Page 5

 CHAPTER 3: Process Management
3.1 What is a Process?
A process is a program in execution. A program is a passive entity (executable file on disk); a process is an
active entity with its own resources, state, and execution context.

Process memory layout (4 segments):

• Text (Code) Segment: Compiled machine instructions. Read-only, shared among instances.
• Data Segment: Initialised global and static variables.
• BSS Segment: Uninitialised global/static variables (zeroed at start).
• Stack: Local variables, function parameters, return addresses. Grows downward.
• Heap: Dynamic allocation (malloc/new). Grows upward. Between stack and BSS.

3.2 Process States

A process transitions through states during its lifetime:

New Process being created. PCB allocated; resources requested.

Ready Process loaded in memory, waiting for CPU. In the ready queue.

Running Instructions being executed. Only one process per CPU core at a time.

Waiting (Blocked) Waiting for an event (I/O completion, signal, resource). Not using CPU.

Terminated Process has finished execution. PCB deallocated after parent reads exit status.

Suspended Ready Ready but swapped out to disk (swapping OS).

Suspended Blocked Waiting and swapped out to disk.

State transitions:

• New → Ready: Admitted by long-term scheduler.

• Ready → Running: Dispatched by short-term scheduler.
• Running → Waiting: I/O request or wait event.
• Waiting → Ready: I/O completion or event.
• Running → Ready: Preempted (time quantum expired).
• Running → Terminated: Process exits.

3.3 Process Control Block (PCB)

Each process is represented by a PCB (also called Task Control Block). The PCB stores all information needed
to manage a process and to save/restore its state.

Process ID (PID) Unique identifier for the process.

Process State Current state: new, ready, running, waiting, terminated.

Program Counter (PC) Address of next instruction to execute.

CPU Registers All register values (accumulator, index regs, stack pointer, etc.).

OS & Unix/Linux Notes Page 6

 Memory Limits Base and limit registers, page tables, segment tables.

I/O Status List of open files, I/O devices allocated.

Accounting Info CPU time used, clock time, time limits, account numbers.

Scheduling Info Priority, scheduling queue pointers, scheduling parameters.

3.4 Process Creation & Termination

3.4.1 Process Creation (Unix/Linux)

pid_t pid = fork(); // create child process
if (pid == 0) {
// Child process
execl("/bin/ls", "ls", "-l", NULL); // replace with new program
} else if (pid > 0) {
// Parent process
wait(NULL); // wait for child to finish
} else {
// fork() failed
perror("fork");
}

• fork(): Creates an exact copy (child) of the calling process. Returns PID of child to parent, 0 to child.
• exec(): Replaces process image with a new program. Does not create new process.
• copy-on-write (COW): Child shares parent's pages until either modifies them — efficient fork().
• Zombie process: Child has terminated but parent hasn't called wait(). PCB still in process table.
• Orphan process: Parent terminated before child. Adopted by init/systemd (PID 1).

3.4.2 Process Termination

• Normal: exit() system call. Exit status code passed to parent via wait().
• Abnormal: killed by signal (SIGKILL, SIGSEGV), or killed by parent via kill().
• Cascading termination: Some OS kill all children when parent terminates.

3.5 Inter-Process Communication (IPC)

Shared Memory Processes map same physical memory region. Fast — no kernel involvement
after setup. Producer-consumer problem. shmget(), shmat().

Message Passing Processes exchange messages via kernel. send(msg) and receive(msg). Safer
but slower. Pipes, message queues, sockets.

Pipes Unidirectional byte stream. Anonymous pipe: between related processes.

Named pipe (FIFO): between any processes. pipe() system call.

Signals Asynchronous notification. kill(pid, sig). SIGTERM=15, SIGKILL=9, SIGINT=2,

SIGSEGV=11.

Sockets Full-duplex communication. Works across networks. TCP/UDP. socket(), bind(),

connect().

Semaphores Synchronisation primitive. Not for data transfer. sem_wait(), sem_post().

OS & Unix/Linux Notes Page 7

 CHAPTER 4: CPU Scheduling
4.1 Scheduling Concepts
CPU scheduling determines which process gets the CPU next. The short-term scheduler (dispatcher) selects
from ready queue processes. Scheduling happens when a process:

• Switches from running to waiting (I/O request).

• Switches from running to ready (interrupt/preemption).
• Switches from waiting to ready (I/O completion).
• Terminates.
Types of scheduling:

• Non-preemptive: CPU not taken from process until it voluntarily yields. Simpler but poor responsiveness.
• Preemptive: CPU can be taken from running process. Modern OS default. Requires synchronisation.

4.2 Scheduling Criteria

CPU Utilisation Keep CPU as busy as possible. Target: 40–90%.

Throughput Number of processes completed per time unit.

Turnaround Time Time from submission to completion = completion - arrival time.

Waiting Time Time spent in ready queue. Turnaround - burst - I/O time.

Response Time Time from submission to first response. Important for interactive systems.

Fairness Each process gets fair share of CPU.

4.3 Scheduling Algorithms

4.3.1 First-Come First-Served (FCFS)

• Processes served in arrival order. Non-preemptive.
• Simple but suffers from convoy effect: short processes wait behind long ones.
• Poor for interactive systems. Good for batch systems.
Example: P1(24ms), P2(3ms), P3(3ms) arrive at t=0
Gantt: |P1(0-24)|P2(24-27)|P3(27-30)|
Avg Waiting: (0+24+27)/3 = 17ms

4.3.2 Shortest Job First (SJF)

• Select process with shortest next CPU burst. Optimal for minimum average waiting time.
• Non-preemptive: Once started, runs to completion of burst.
• Preemptive version = Shortest Remaining Time First (SRTF).
• Problem: Cannot know future burst length. Estimate using exponential averaging: tn+1 = alpha*tn +
(1-alpha)*Tn.

4.3.3 Priority Scheduling

• Each process assigned a priority number. CPU given to highest priority.
• Can be preemptive or non-preemptive.

OS & Unix/Linux Notes Page 8

 • Problem: Starvation — low priority processes may never execute.
• Solution: Ageing — gradually increase priority of waiting processes.
• SJF is priority scheduling where priority = predicted next burst (inverse).

4.3.4 Round Robin (RR)

• Each process gets a fixed time quantum (typically 10–100ms).
• Preemptive. After quantum expires, process is preempted and goes to back of ready queue.
• Good for time-sharing. Response time bounded by (n-1)*q for n processes.
• Large q → behaves like FCFS. Small q → many context switches, overhead.
• Rule of thumb: 80% of CPU bursts should be shorter than time quantum.

4.3.5 Multilevel Queue Scheduling

• Ready queue partitioned into multiple queues by process type.
• Foreground (interactive): Round Robin. Background (batch): FCFS.
• Fixed priority between queues — foreground always scheduled first.
• Time slice between queues: e.g., 80% foreground, 20% background.

4.3.6 Multilevel Feedback Queue (MLFQ)

• Most general and widely used algorithm (Linux CFS inspired by it).
• Processes can move between queues based on behaviour.
• CPU-bound processes sink to lower priority queues over time.
• I/O-bound processes stay in high priority queues (burst often < quantum).
• Ageing implemented by moving processes upward over time.

Algorithm Type Starvation Best For Avg Wait

FCFS Non-preemptive No Batch High (convoy)

SJF Non-preemptive Yes Known bursts Minimum

SRTF Preemptive Yes Known bursts Minimum

Priority Both Yes Critical tasks Varies

Round Robin Preemptive No Time-sharing Medium

MLFQ Preemptive No* General purpose Low

OS & Unix/Linux Notes Page 9

 CHAPTER 5: Thread Management
5.1 Threads vs Processes
A thread is the smallest unit of CPU execution. A process can contain multiple threads, all sharing the process's
address space, open files, and OS resources.

Process Independent execution unit. Own address space, PCB, resources. Heavy to
create/switch.

Thread Lightweight process. Shares code, data, heap with sibling threads. Own stack,
registers, PC.

Thread creation Faster than process creation (no new address space). ~10-100x faster.

Context switch Thread switch within same process: faster (no TLB flush, address space stays
same).

5.2 Benefits of Multithreading

• Responsiveness: UI thread stays responsive while worker thread does computation.
• Resource Sharing: Threads share process memory — no IPC overhead for communication.
• Economy: Creating and switching threads is cheaper than processes.
• Scalability: Threads can run truly in parallel on multi-core CPUs.

5.3 Thread Models

Many-to-One Many user threads mapped to one kernel thread. Blocking syscall blocks all
threads. Green threads (early Java).

One-to-One Each user thread maps to one kernel thread. True parallelism but overhead.
Linux, Windows, macOS.

Many-to-Many Many user threads multiplexed over many kernel threads. Best of both.
Complex. Solaris.

Two-Level Variant of M:M — allows binding specific threads to kernel threads. Solaris 2.

5.4 POSIX Threads (pthreads)

#include

void* worker(void* arg) {

printf("Thread ID: %lu\n", pthread_self());
return NULL;
}

int main() {
pthread_t tid;
pthread_create(&tid;, NULL, worker, NULL); // create
pthread_join(tid, NULL); // wait for thread
return 0;
}

OS & Unix/Linux Notes Page 10

 pthread_create() Create a new thread with specified function.

pthread_join() Wait for thread termination; collect return value.

pthread_exit() Terminate calling thread.

pthread_self() Return calling thread's ID.

pthread_cancel() Request cancellation of another thread.

pthread_mutex_*() Mutex lock/unlock for synchronisation.

5.5 Thread Issues

• fork() semantics: Does fork() duplicate all threads or just calling thread? Linux: duplicates only calling thread.
• Signal handling: Which thread receives signal? POSIX: process-directed signals delivered to first thread that
doesn't block them.
• Thread-local storage (TLS): Each thread has private data even for global variables. __thread keyword in C.
• Thread pools: Pre-create pool of threads; assign tasks as needed. Avoids overhead of thread creation per
request.
• Daemon threads: Background threads that die when main thread exits.

OS & Unix/Linux Notes Page 11

 CHAPTER 6: Concurrency & Synchronization
6.1 The Critical Section Problem
A critical section is a code segment that accesses shared resources and must not be executed by more than
one process/thread concurrently. The critical section problem is to design a protocol so processes can
cooperate.

Requirements for a correct solution:

• Mutual Exclusion: Only one process in its critical section at a time.

• Progress: If no process is in CS and some wish to enter, selection cannot be postponed indefinitely.
• Bounded Waiting: A limit on the number of times other processes can enter CS after a process requests
entry and before it's granted.

6.2 Race Conditions & Synchronisation

A race condition occurs when the result depends on the relative timing of events across concurrent
threads/processes. Example: two threads both do x++ concurrently — if x=5, the result might be 6 instead of 7
because both read x=5 before writing.

6.3 Synchronisation Mechanisms

6.3.1 Mutex (Mutual Exclusion Lock)

• Binary lock: either locked or unlocked. Only the locking thread can unlock it.
• acquire(): wait until lock free, then lock. release(): unlock.
• Spin lock: busy-waits (wastes CPU). Better for very short critical sections on multi-core.
• Blocking mutex: thread sleeps if lock unavailable. Better for longer CS.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

// Thread 1 & 2:
pthread_mutex_lock(&mutex;); // acquire
// critical section
counter++; // protected shared variable
pthread_mutex_unlock(&mutex;); // release

6.3.2 Semaphores
A semaphore S is an integer variable accessed only through two atomic operations: wait(S) (P operation) and
signal(S) (V operation).
wait(S): while (S <= 0) ; // busy wait OR sleep
S--;

signal(S): S++;

// Binary semaphore (mutex): S initialised to 1

// Counting semaphore: S = number of resources available

• Binary semaphore: Like mutex but any thread can signal it.
• Counting semaphore: Controls access to N instances of a resource.
• Deadlock risk: Incorrect use of semaphores can cause deadlock or starvation.

OS & Unix/Linux Notes Page 12

 6.3.3 Monitors
• High-level synchronisation construct. Only one process active in monitor at a time.
• Condition variables: wait() releases monitor lock and sleeps. signal() wakes one waiter.
• Java's synchronized methods implement monitors. Python [Link].

6.4 Classic Synchronisation Problems

Producer-Consumer (Bounded Buffer) Problem
• Producer adds items to buffer; consumer removes them. Buffer has fixed size N.
• Need: mutex (protect buffer), empty (count empty slots, init=N), full (count full slots, init=0).
• Producer: wait(empty) → wait(mutex) → add → signal(mutex) → signal(full).
• Consumer: wait(full) → wait(mutex) → remove → signal(mutex) → signal(empty).

Readers-Writers Problem
• Multiple readers can read simultaneously. Writers need exclusive access.
• First readers-writers: no reader waits unless writer has lock. Writers may starve.
• Second readers-writers: once writer ready, no new readers start. Readers may starve.
• Solution: Reader-writer locks (pthread_rwlock_t).

Dining Philosophers Problem

• N philosophers sit at round table. N forks between them. Need 2 forks to eat.
• Deadlock if all pick up left fork simultaneously.
• Solutions: Allow at most N-1 at table; pick both forks atomically; asymmetric (odd picks left first, even picks
right first).

OS & Unix/Linux Notes Page 13

 CHAPTER 7: Deadlocks
7.1 Deadlock Definition
A deadlock is a situation where a set of processes are blocked, each waiting for a resource held by another
process in the set — a circular wait with no way out.

7.2 Four Necessary Conditions (Coffman Conditions)

All four must hold simultaneously for deadlock to occur. Preventing any one prevents deadlock.

1. Mutual Exclusion At least one resource held in non-sharable mode (only one process at a time).

2. Hold and Wait Process holds at least one resource and is waiting to acquire additional
resources held by others.

3. No Preemption Resources cannot be forcibly taken from a process; only released voluntarily.

4. Circular Wait Set of processes {P0,P1,...,Pn} where P0 waits for P1, P1 for P2, ..., Pn waits for
P0.

7.3 Resource Allocation Graph (RAG)

• Vertices: Processes (circles) and Resource types (rectangles with dots for instances).
• Request edge: P → R (process wants resource).
• Assignment edge: R → P (resource instance assigned to process).
• If graph has no cycle: no deadlock.
• If graph has cycle with single-instance resources: deadlock.
• If graph has cycle with multi-instance resources: may or may not be deadlock.

7.4 Deadlock Handling Strategies

7.4.1 Deadlock Prevention

Ensure at least one Coffman condition cannot hold:

• Mutual Exclusion: Not always preventable (some resources inherently non-sharable).

• Hold & Wait: Require process to request ALL resources at once before starting. OR release all before new
request. Low utilisation.
• No Preemption: If process requests resource not available, release all held resources. Good for CPU
registers, not for printers.
• Circular Wait: Impose total ordering on resource types. Process requests in increasing order. Most practical
prevention.

7.4.2 Deadlock Avoidance

OS makes resource allocation decision dynamically to ensure system never enters unsafe state. Requires
advance knowledge of maximum resource needs.

• Safe state: There exists a safe sequence where all processes can complete.
• Unsafe state: No such sequence exists — deadlock possible (but not certain).
• Banker's Algorithm (Dijkstra): Check if granting a request keeps system in safe state.

OS & Unix/Linux Notes Page 14

 Banker's Algorithm Data Structures:
Allocation[n][m]: resources currently allocated to each process
Max[n][m]: max resources each process may need
Available[m]: currently available resources
Need = Max - Allocation

Safety Algorithm:
Work = Available; Finish[i] = false for all i
Find i: !Finish[i] AND Need[i] <= Work
Work += Allocation[i]; Finish[i] = true
If all Finish[i] = true: SAFE STATE

7.4.3 Deadlock Detection & Recovery

• Detection: Allow deadlocks; detect them using wait-for graph (single instance) or detection algorithm
(multi-instance). Run periodically or when CPU utilisation drops.
• Recovery — Process Termination: Kill all deadlocked processes (costly). OR kill one at a time until
deadlock broken.
• Recovery — Resource Preemption: Preempt resources from processes. Issues: rollback to safe state,
starvation (same process always preempted).

7.5 Deadlock Ignorance (Ostrich Algorithm)

• Pretend deadlocks never occur. Let them crash the system.
• Used by Windows and Linux — deadlocks are rare and detection/prevention is costly.
• Acceptable for desktop OS; unacceptable for real-time/mission-critical systems.

OS & Unix/Linux Notes Page 15

 CHAPTER 8: Memory Management
8.1 Memory Hierarchy & Goals
• Registers → L1 Cache → L2 Cache → L3 Cache → RAM → SSD → HDD (speed decreases, size increases).
• OS manages RAM — allocates to processes, protects boundaries, enables sharing.
• Goals: Relocation, Protection, Sharing, Logical/Physical organisation.

8.2 Memory Allocation

8.2.1 Contiguous Memory Allocation

• Fixed partitioning: Memory divided into fixed-size partitions. Internal fragmentation (unused space within
partition).
• Variable partitioning: Partitions fit process size. External fragmentation (free space scattered).
• First Fit: Allocate first hole big enough. Fast.
• Best Fit: Allocate smallest hole that fits. Minimises wasted space but slow.
• Worst Fit: Allocate largest hole. Leaves largest remnant.
• Compaction: Shuffle memory to consolidate free space. Expensive.

8.2.2 Paging
Paging eliminates external fragmentation by dividing physical memory into fixed-size frames and logical
memory into same-size pages. OS maintains a page table per process mapping logical pages to physical
frames.

• Page size typically 4KB (can be 2MB, 1GB with huge pages).
• Logical address = Page number (p) | Page offset (d). Physical address = Frame number | offset.
• Internal fragmentation: average half-page wasted per process.
• No external fragmentation — any free frame can be used.
• TLB (Translation Lookaside Buffer): Hardware cache for page table entries. Hit ratio ~99%. Without TLB: 2
memory accesses per access (page table + data).
• Multi-level paging: Page table too large for one structure. Two-level (x86): PD → PT → Frame. Four-level
(x86-64).
• Inverted page table: One entry per physical frame. Reduces memory but slow lookup. IBM PowerPC.

8.2.3 Segmentation
• Memory viewed as a collection of named segments of variable size.
• Logical address = . Segment table: base + limit per segment.
• External fragmentation exists. Supports natural program structure (code, stack, heap as separate segments).
• Intel x86 uses segmentation + paging together.

8.3 Virtual Memory

Virtual memory allows processes to run even if not entirely in RAM. Pages loaded on demand.

8.3.1 Demand Paging

• Load pages only when needed (page fault). Invalid bit in page table entry indicates page not in RAM.

OS & Unix/Linux Notes Page 16

 • Page fault handling: (1) Check page table — invalid ref → abort. (2) Find free frame. (3) Load page from
disk. (4) Update page table. (5) Restart instruction.
• Effective Access Time (EAT): EAT = (1-p)*mem_access + p*(page_fault_time). p=page fault rate.

8.3.2 Page Replacement Algorithms

FIFO Replace oldest page in memory. Simple. Belady's Anomaly: more frames can
cause more faults.

Optimal (OPT) Replace page not used for longest time in future. Not implementable —
theoretical optimal.

LRU Replace least recently used page. No Belady's anomaly. Approx: counter per
page or stack.

LRU Approximation Reference bit per page. Clock algorithm (second-chance): circular queue of
pages.

LFU Replace least frequently used page. Counters shift right periodically to age.
■ Note: Linux uses a variant of the clock algorithm (active/inactive LRU lists) for page replacement.

8.3.3 Thrashing
• Process spends more time paging than executing — page fault rate very high.
• Cause: too many processes with too little memory; working set doesn't fit in RAM.
• Working set model: W(t, delta) = set of pages used in last delta time units. OS ensures working set fits in
memory.
• Solution: Reduce degree of multiprogramming; increase RAM; use working set tracking.

8.4 Primary Memory (RAM)

DRAM Dynamic RAM — uses capacitors; needs refresh every ~64ms. Cheap. Used for
main memory.

SRAM Static RAM — uses flip-flops; no refresh; fast. Expensive. Used for CPU caches.

ROM/PROM/EPROM Read-only or once-writable. Stores firmware (BIOS/UEFI).

NVRAM/Flash Non-volatile. SSDs, USB drives. Slower writes; limited write cycles.

OS & Unix/Linux Notes Page 17

 CHAPTER 9: File System Management
9.1 File Concepts
A file is a named collection of related data stored on secondary storage. The OS provides a uniform logical view
of storage through the file system.

File Attributes Name, identifier (inode number), type, location, size, protection, timestamps.

File Operations create, delete, open, close, read, write, seek, truncate, rename.

File Types Regular files, directories, symbolic links, device files (block/char), pipes (FIFO),
sockets.

File Structure Byte sequence (Unix — simplest), record sequence, tree (keyed records).

9.2 Directory Structure

Single-Level One directory for all users. Name conflicts with multiple users.

Two-Level Separate directory per user. No sharing between users.

Tree-Structured Generalised hierarchy. Absolute and relative paths. Current directory.

Acyclic Graph Allows shared subdirectories/files via hard links and symbolic links.

General Graph Cycles possible — need garbage collection. Avoided in practice.

9.3 File System Implementation

9.3.1 Disk Layout

• Boot Block: Contains bootstrap code to load OS.
• Superblock: File system metadata — size, free blocks count, inode count, magic number.
• Inode Table: Array of inodes; one per file.
• Data Blocks: Actual file content.

9.3.2 Inodes (Unix)

• Inode stores all file metadata except the filename.
• Contains: file type, permissions, owner UID/GID, size, timestamps (atime/mtime/ctime), link count, block
pointers.
• Block pointers: 12 direct, 1 single indirect, 1 double indirect, 1 triple indirect.
• Direct: 12 * 4KB = 48KB. Single indirect: 1024 * 4KB = 4MB. Double indirect: 1024^2 * 4KB = 4GB.

9.3.3 Disk Space Allocation Methods

Contiguous Store file in consecutive blocks. Simple; fast sequential access. External
fragmentation; file size fixed.

Linked List Each block has pointer to next. No fragmentation. Random access slow O(n).

FAT File Allocation Table in memory — linked list variant. MS-DOS, USB drives. Fast
random access with FAT in RAM.

OS & Unix/Linux Notes Page 18

 Indexed (Unix inode) Index block holds all block pointers. Random access O(1). Supports large files
via multi-level indexing.

9.3.4 Free Space Management

• Bit vector (bitmap): One bit per block. 0=free, 1=allocated. Simple and efficient.
• Linked list: Chain all free blocks. Traversal expensive.
• Grouping: First free block stores n free block addresses. Last is address of block with next n.
• Counting: Store (first free block, count of consecutive free blocks).

9.4 File System Types

ext2 Linux — no journalling. Fast, simple. Risk of corruption on unclean shutdown.

ext3 ext2 + journalling. Three modes: writeback, ordered, journal.

ext4 Current Linux default. Large files, extents, delayed allocation, journal
checksumming.

XFS High performance journalling FS. Excellent for large files and parallel I/O.

Btrfs B-tree FS. Copy-on-write, snapshots, RAID, checksumming. Next-gen Linux FS.

FAT32 Wide compatibility. No permissions, 4GB file size limit.

NTFS Windows. Journalling, ACLs, encryption, compression, sparse files.

APFS Apple. Copy-on-write, snapshots, encryption, SSD-optimised.

tmpfs In-memory file system. /tmp, /dev/shm on Linux.

procfs Virtual FS. /proc — exposes kernel/process information as files.

sysfs Virtual FS. /sys — exposes kernel device tree.

9.5 File System Interface

• VFS (Virtual File System): Abstraction layer. Applications use standard calls; VFS dispatches to specific FS
driver.
• Mounting: mount /dev/sdb1 /mnt — attaches a file system to a directory.
• Hard link: Directory entry pointing to same inode. Same inode number. Cannot span file systems. ln file link.
• Symbolic (soft) link: File containing path to another file. Can span FS. Broken if target deleted. ln -s.
• File descriptor: Per-process integer index into open file table. 0=stdin, 1=stdout, 2=stderr.

OS & Unix/Linux Notes Page 19

 CHAPTER 10: Secondary Storage & I/O Systems
10.1 Disk Structure
Platter Circular disk coated with magnetic material.

Track Concentric circle on a platter surface.

Sector Smallest addressable unit on a track. Typically 512 bytes or 4096 bytes.

Cylinder Set of tracks at same radial position across all platters.

Head Read/write head per platter surface.

Access time = Seek time + Rotational latency + Transfer time

10.2 Disk Scheduling Algorithms

FCFS Service requests in order received. Simple; high seek time.

SSTF Shortest Seek Time First. Service closest request. Possible starvation.

SCAN (Elevator) Move head in one direction, servicing requests; reverse at end. Fair.

C-SCAN Circular SCAN. Move in one direction; jump to beginning. More uniform wait
time.

LOOK Like SCAN but reverses when no more requests in direction, not at physical end.

C-LOOK Circular LOOK. Best for uniform load.

10.3 I/O Systems

Programmed I/O CPU polls device status register continuously. Wastes CPU cycles.

Interrupt-Driven Device interrupts CPU when ready. CPU does other work meanwhile.

DMA (Direct Memory DMA controller manages data transfer between device and memory. CPU
Access) initiates and is interrupted only on completion. Best for bulk transfers.

Memory-Mapped I/O Device registers mapped to memory addresses. CPU uses normal load/store
instructions.

Port-Mapped I/O Separate I/O address space. Special IN/OUT instructions (x86).

10.4 I/O Software Layers

• Hardware: Physical device.
• Device drivers: OS kernel modules for specific devices.
• Device-independent I/O software: Uniform interface, naming, protection, buffering, error reporting.
• User-space I/O: Libraries (stdio), spooling (print queues).

10.5 RAID (Redundant Array of Independent Disks)

OS & Unix/Linux Notes Page 20

 RAID 0 Striping — data split across disks. Max performance, no redundancy. Any disk
failure = data loss.

RAID 1 Mirroring — duplicate on 2+ disks. Full redundancy. 50% space efficiency.

RAID 5 Striping with distributed parity. N-1 disks capacity. Tolerates 1 disk failure.
Common in NAS.

RAID 6 Like RAID 5 but dual parity. Tolerates 2 failures.

RAID 10 RAID 1+0. Mirror sets then stripe. High performance + redundancy. Needs 4+
disks.

OS & Unix/Linux Notes Page 21

 CHAPTER 11: Protection & Security
11.1 Goals of Protection & Security
• Protection: Mechanisms to control access of processes and users to OS resources.
• Security: Defend against external threats — unauthorized access, malicious code, denial of service.
• Principle of Least Privilege: every process/user gets minimum access needed.
• Principle of Fail-Safe Defaults: default is no access; explicitly grant.
• Principle of Complete Mediation: every access checked.

11.2 Access Matrix

• Rows = domains (users/processes). Columns = objects (files, devices, processes).
• Matrix[domain][object] = set of allowed operations (read, write, execute, own).
• Access Control List (ACL): Store matrix by column — each object has list of (domain, rights).
• Capability list: Store matrix by row — each domain has list of (object, rights). Harder to revoke.
• Unix/Linux uses simplified ACL: owner/group/others × read/write/execute (9 bits + setuid/setgid/sticky).

11.3 Common Security Threats

Buffer Overflow Write beyond array bounds to overwrite return address. Execute injected
shellcode. Fixed: ASLR, stack canaries, NX bit.

Privilege Escalation Gain higher privileges than granted. Exploit setuid bugs, kernel vulnerabilities.

Trojan Horse Malicious code hidden in legitimate program.

Rootkit Hides malicious activity by modifying OS. Often replaces kernel modules.

SQL Injection Inject SQL into input fields to manipulate database.

DDoS Distributed Denial of Service — flood target with traffic to exhaust resources.

Man-in-the-Middle Intercept and possibly alter communications between two parties.

11.4 Cryptography in OS Security

• Symmetric encryption: Same key for encrypt/decrypt. AES-256. Fast. Key distribution problem.
• Asymmetric encryption: Public/private key pair. RSA, ECC. Slower. Solves key distribution.
• Hash functions: One-way. SHA-256. Password storage, integrity verification.
• Digital signatures: Sign with private key; verify with public key. Authentication + non-repudiation.
• TLS/SSL: Combines asymmetric (key exchange) + symmetric (bulk data) + hashes (integrity).

11.5 Linux Security Modules (LSM)

• SELinux: Security-Enhanced Linux. Mandatory Access Control (MAC). Policy-based.
• AppArmor: Profile-based MAC. Simpler than SELinux. Used by Ubuntu.
• seccomp: Filter system calls available to a process. Used by Docker, Chrome.
• Capabilities: Fine-grained root privileges. Instead of all-or-nothing root, split into 38+ capabilities.
CAP_NET_BIND_SERVICE, CAP_SYS_ADMIN.

OS & Unix/Linux Notes Page 22

 CHAPTER 12: Linux – Process Management
12.1 Linux Process Model
Linux treats almost everything as a process (or task). The kernel uses task_struct (the Linux PCB) to represent
each process/thread. Linux uses clone() under the hood for both fork() and pthread_create().

12.2 Viewing Processes

ps List current processes. Snapshot.

ps aux All processes, all users, with BSD-style detailed output.

ps -ef All processes, full format (PPID, CMD, etc.).

top Interactive real-time process viewer. CPU, memory, load average.

htop Enhanced top. Colour-coded, mouse support. Must install.

pgrep nginx Find PID(s) of processes named 'nginx'.

pstree Show processes as tree (parent-child hierarchy).

pidof bash Get PID of running program by name.

12.3 Managing Processes

kill PID Send SIGTERM (15) to process — graceful termination request.

kill -9 PID Send SIGKILL (9) — force kill, cannot be caught or ignored.

kill -l List all signal names and numbers.

killall nginx Kill all processes named 'nginx' by name.

pkill -u user Kill all processes owned by 'user'.

nice -n 10 cmd Run cmd with lower priority (nice value 10, range -20 to 19).

renice 5 -p PID Change nice value of running process.

nohup cmd & Run cmd immune to hangup; continues after logout. Output to [Link].

bg Resume stopped job in background.

fg %1 Bring job 1 to foreground.

jobs List current shell's background/stopped jobs.

Ctrl+Z Suspend (stop) foreground process — sends SIGTSTP.

Ctrl+C Interrupt foreground process — sends SIGINT.

12.4 Process Priority & Scheduling in Linux

• Linux CFS (Completely Fair Scheduler) since kernel 2.6.23.
• Uses red-black tree keyed by virtual runtime (vruntime). Leftmost node = next to run.
• Nice value: -20 (highest priority) to +19 (lowest). Default 0.
• Real-time scheduling classes: SCHED_FIFO, SCHED_RR (priority 1-99 above normal).

OS & Unix/Linux Notes Page 23

 • /proc/PID/status — view process details. /proc/PID/stat — scheduler info.

12.5 Phases of a Process (Linux)

TASK_RUNNING Process is running or runnable (in run queue).

TASK_INTERRUPTIBLE Sleeping, waiting for event. Can be woken by signal.

TASK_UNINTERRUPTIB Sleeping, waiting for I/O. Cannot be interrupted by signal. Shows as 'D' in ps.
LE

TASK_STOPPED Stopped by SIGSTOP or SIGTSTP. Resumes with SIGCONT.

TASK_ZOMBIE Process finished but parent hasn't called wait(). Shows as 'Z'.

EXIT_DEAD Being removed from process table.

■ Note: In 'ps aux', the STAT column shows: R=running, S=sleeping, D=disk wait, Z=zombie, T=stopped,
s=session leader, +=foreground.

OS & Unix/Linux Notes Page 24

 CHAPTER 13: Linux File Permissions
13.1 Permission Model
Every Linux file has an owner (user), a group, and permissions for three categories: owner (u), group (g),
others (o). Each has three bits: read (r=4), write (w=2), execute (x=1).
$ ls -l [Link]
-rwxr-xr-- 1 alice devs 4096 Mar 10 09:00 [Link]
|||||||||||
|uuu|ggg|ooo
type=regular(-), owner=rwx(7), group=r-x(5), others=r--(4)

File type char - = regular, d = directory, l = symlink, c = char device, b = block device, p = pipe,
s = socket

Octal notation rwxr-xr-- = 754. Each octet is sum of r(4)+w(2)+x(1).

Directory x bit Execute on directory means 'search' — needed to cd into or access files within.

13.2 Changing Permissions and Ownership

chmod 755 file Set owner=rwx(7), group=r-x(5), others=r-x(5).

chmod u+x file Add execute for owner.

chmod go-w file Remove write from group and others.

chmod a+r file Add read for all (a = all = ugo).

chmod -R 644 dir/ Recursively set permissions.

chown alice file Change owner to alice.

chown alice:devs file Change owner to alice, group to devs.

chgrp devs file Change group to devs.

chown -R alice:devs dir/ Recursively change owner/group.

13.3 Special Permission Bits

setuid (SUID) 4xxx When set on executable: runs with owner's privileges. /usr/bin/passwd has SUID
root — allows changing password. ls shows 's' in owner execute bit.

setgid (SGID) 2xxx On executable: runs with group's privileges. On directory: new files inherit
directory's group. Useful for shared project dirs.

Sticky Bit 1xxx On directory: users can only delete their own files even if they have write
permission on directory. /tmp uses sticky bit. ls shows 't' in others execute bit.
$ ls -l /usr/bin/passwd /tmp
-rwsr-xr-x root root passwd # SUID set (s in owner exec)
drwxrwxrwt root root /tmp # sticky bit (t in others exec)

13.4 umask
• umask defines default permission restrictions when creating files.

OS & Unix/Linux Notes Page 25

 • Default file permissions: 666. Default dir permissions: 777.
• Actual permissions = default - umask. Common umask: 022 → files=644, dirs=755.
• Set with: umask 022. View with: umask.

13.5 Access Control Lists (ACLs)

• Extends basic Unix permissions to allow per-user/per-group permissions.
• getfacl file — view ACL. setfacl -m u:bob:rw file — give bob read+write.
• setfacl -m g:sales:r file — give sales group read. setfacl -x u:bob file — remove ACL entry.

OS & Unix/Linux Notes Page 26

 CHAPTER 14: Linux Startup and Shutdown
14.1 The Boot Sequence in Detail
1. BIOS/UEFI Initialise hardware. POST. Locate bootable device via boot order.

2. MBR/GPT First 512 bytes of disk (MBR) or GPT header. Loads bootloader stage 1.

3. GRUB2 Stage 1 Loads GRUB2 stage 1.5 from post-MBR gap or EFI partition.

4. GRUB2 Stage 2 Displays boot menu. Loads kernel image (vmlinuz) + initrd into RAM.

5. Kernel Init Kernel decompresses itself. Detects CPU, memory. Mounts initramfs.

6. initramfs Temporary root FS in RAM. Contains drivers needed to mount real root. Runs
/init script.

7. Real Root FS Kernel mounts real root filesystem on /. Transitions from initramfs.

8. systemd/init Kernel executes /sbin/init (PID=1). systemd reads unit files; starts services.

9. Runlevel/Target systemd activates target ([Link], [Link]). Services start.

10. Login getty on TTYs or display manager (gdm, lightdm) presents login.

14.2 systemd
• Modern init system, replaces SysVinit. Uses unit files (not shell scripts).
• Parallel service startup — much faster boot than SysVinit.
• Dependency-based: service starts only when its dependencies are met.

systemctl start nginx Start a service.

systemctl stop nginx Stop a service.

systemctl restart nginx Stop then start.

systemctl reload nginx Reload config without stopping.

systemctl enable nginx Enable at boot (create symlink in target).

systemctl disable nginx Disable at boot.

systemctl status nginx Show service status, recent log.

systemctl list-units List all loaded units.

systemctl get-default Show default target (boot target).

systemctl isolate Switch to multi-user target (like runlevel 3).

[Link]

journalctl -u nginx View logs for nginx service.

journalctl -f Follow (tail) system journal.

journalctl --since today Show today's logs.

14.3 systemd Targets (Runlevel Equivalents)

OS & Unix/Linux Notes Page 27

 systemd Target SysV Runlevel Description

[Link] 0 Halt system

[Link] 1 Single user mode

[Link] 3 Multi-user, no GUI

[Link] 5 Multi-user with GUI

[Link] 6 Reboot

[Link] — Emergency shell (read-only root)

14.4 Shutdown Commands

shutdown -h now Halt system immediately.

shutdown -h +10 Halt in 10 minutes.

shutdown -r now Reboot immediately.

shutdown -c Cancel pending shutdown.

poweroff Halt and power off.

reboot Reboot system.

halt Halt system (may not power off).

systemctl poweroff Power off via systemd.

systemctl reboot Reboot via systemd.

init 0 SysV equivalent of shutdown.

OS & Unix/Linux Notes Page 28

 CHAPTER 15: Unix Operating System Overview
15.1 Unix Philosophy
• Write programs that do one thing and do it well.
• Write programs to work together (pipelines, text streams).
• Write programs that handle text streams — the universal interface.
• Everything is a file — devices, processes, sockets all accessed via file operations.
• Small, sharp tools combined with pipes rather than monolithic programs.

15.2 Unix Architecture

Hardware CPU, memory, I/O devices, storage.

Kernel Core of Unix. Manages processes, memory, file system, I/O. Runs in privileged
mode.

Shell Command interpreter between user and kernel. sh, csh, ksh, bash, zsh.

System Libraries C library, math library — wrappers around system calls.

User Programs Applications that use system calls and library functions.

15.3 Unix File System Hierarchy

/ Root directory. Top of entire filesystem hierarchy.

/bin Essential user binaries (ls, cp, mv, sh). Available in single-user mode.

/sbin System binaries for root (fsck, ifconfig, init).

/etc System-wide configuration files (passwd, fstab, hosts, cron).

/home User home directories (/home/alice, /home/bob).

/root Root user's home directory.

/var Variable data: logs (/var/log), mail, spool, databases.

/tmp Temporary files. Cleared on reboot.

/usr Secondary hierarchy. /usr/bin (user programs), /usr/lib, /usr/share.

/lib Shared libraries needed by /bin and /sbin.

/dev Device files. /dev/sda (disk), /dev/null, /dev/random, /dev/tty.

/proc Virtual FS. Process and kernel info: /proc/PID, /proc/meminfo, /proc/cpuinfo.

/mnt Mount points for temporary mounts.

/opt Optional/third-party software packages.

/boot Boot loader files, kernel image, initrd.

OS & Unix/Linux Notes Page 29

 CHAPTER 16: Unix/Linux Commands
16.1 Navigation & File Management
pwd Print working directory — show current path.

ls List directory contents.

ls -l Long format: permissions, owner, size, date.

ls -la Long format including hidden files (starting with .).

ls -lh Long format with human-readable sizes (KB, MB).

ls -lt Sort by modification time, newest first.

cd /path Change to absolute path.

cd .. Go up one directory level.

cd ~ Go to home directory.

cd - Go to previous directory.

mkdir dir Create directory.

mkdir -p a/b/c Create nested directories (no error if exists).

rmdir dir Remove empty directory.

rm file Delete file.

rm -r dir/ Delete directory and contents recursively.

rm -rf dir/ Force recursive delete (no prompts). DANGEROUS.

mv src dst Move or rename file/directory.

cp src dst Copy file.

cp -r src/ dst/ Copy directory recursively.

cp -p src dst Preserve permissions, timestamps, owner.

touch file Create empty file or update timestamps.

ln file link Create hard link.

ln -s target link Create symbolic (soft) link.

find / -name f Find files named 'f' starting from /.

locate filename Fast file search using updatedb database.

which cmd Show full path of command executable.

whereis cmd Find binary, source, and man page for command.

file filename Determine file type by content inspection.

stat file Detailed file info: inode, size, timestamps, permissions.

du -sh dir/ Disk usage of directory, human-readable summary.

df -h Disk free space on all mounted file systems.

OS & Unix/Linux Notes Page 30

 16.2 Absolute and Relative Pathnames
• Absolute path: Starts from root /. Always the same regardless of current directory. E.g.,
/home/alice/docs/[Link]
• Relative path: Relative to current working directory. E.g., ../docs/[Link] or ./[Link]
• . refers to current directory. .. refers to parent directory.
• ~ refers to home directory. ~user refers to user's home.
$ pwd
/home/alice
$ ls ../bob/ # relative: go up, then into bob
$ ls /home/bob/ # absolute: same directory
$ cd ~/Documents # ~ expands to /home/alice

16.3 Viewing File Content

cat file Display entire file content.

cat -n file Display with line numbers.

more file Page through file (forward only). Space=next page, q=quit.

less file Page through file (forward+backward). Better than more.

head file Display first 10 lines.

head -n 20 file Display first 20 lines.

tail file Display last 10 lines.

tail -n 20 file Display last 20 lines.

tail -f file Follow file — display new lines as appended. Great for logs.

wc file Count lines, words, characters.

wc -l file Count lines only.

wc -w file Count words only.

wc -c file Count bytes/characters.

OS & Unix/Linux Notes Page 31

 CHAPTER 17: Copying, Renaming & Deleting Files in Unix
17.1 Copying Files (cp)
cp source destination # basic copy
cp file1 file2 dir/ # copy multiple files to directory
cp -r sourcedir/ destdir/ # recursive (directory)
cp -p file backup # preserve metadata
cp -i file dest # interactive (prompt before overwrite)
cp -u src dst # copy only if src newer than dst
cp -v src dst # verbose (show what is copied)
cp -a srcdir/ dstdir/ # archive: recursive + preserve all attributes

■ Tip: cp -a is the best option for backing up directory trees — preserves permissions, symlinks, timestamps.

17.2 Renaming and Moving Files (mv)

mv oldname newname # rename file
mv [Link] /other/dir/ # move to another directory
mv [Link] /other/dir/[Link] # move and rename simultaneously
mv -i src dst # prompt before overwrite
mv -v src dst # verbose
mv -b src dst # backup overwritten destination
mv dir1 dir2/ # move directory into dir2

17.3 Deleting Files (rm)

rm file # delete file (no recycle bin!)
rm file1 file2 file3 # delete multiple
rm -i file # prompt before each deletion
rm -f file # force delete, no errors
rm -r dir/ # delete directory recursively
rm -rf dir/ # force recursive — very dangerous
rmdir emptydir # remove only if empty

■ Note: There is no recycle bin for rm. Once deleted, files are gone (unless using backups or forensic tools).
Always use rm -i in doubt.

OS & Unix/Linux Notes Page 32

 CHAPTER 18: cat Command in Unix
18.1 The cat Command
cat (concatenate) reads files sequentially and writes them to standard output. It is one of the most versatile Unix
commands.

cat file Display file contents.

cat file1 file2 Concatenate and display both files.

cat file1 file2 > out Concatenate and save to out.

cat -n file Number all output lines.

cat -b file Number non-empty lines only.

cat -s file Squeeze multiple blank lines into one.

cat -A file Show all special chars: $ at line ends, ^I for tabs.

cat -e file Show $ at end of each line (visualise newlines).

cat -t file Show ^I for tab characters.

cat -v file Show non-printing characters in ^ notation.

cat > newfile Create file from keyboard input (Ctrl+D to finish).

cat >> file Append keyboard input to existing file.

cat /dev/null > file Empty a file (truncate to zero bytes).

18.2 cat vs tac vs rev

• cat: Output file lines in normal order.
• tac: Output file lines in reverse order (last line first).
• rev: Reverse each line character by character.
$ echo -e 'line1\nline2\nline3' | tac
line3
line2
line1

$ echo 'hello' | rev

olleh

OS & Unix/Linux Notes Page 33

 CHAPTER 19: Filtering Commands in Unix
19.1 grep – Search Text
grep (Global Regular Expression Print) searches input for lines matching a pattern.

grep 'pat' file Search for pattern in file.

grep -i 'pat' file Case-insensitive search.

grep -n 'pat' file Show line numbers.

grep -v 'pat' file Invert — show lines NOT matching.

grep -c 'pat' file Count matching lines.

grep -l 'pat' *.txt List filenames with matches.

grep -r 'pat' dir/ Recursive search in directory.

grep -w 'word' file Match whole word only.

grep -A 3 'pat' file Show 3 lines After match.

grep -B 3 'pat' file Show 3 lines Before match.

grep -C 3 'pat' file Show 3 lines Context (before+after).

grep -E 'p1|p2' file Extended regex (egrep equivalent).

grep -P 'regex' file Perl-compatible regex (PCRE).

grep '^start' file Lines beginning with 'start'.

grep 'end$' file Lines ending with 'end'.

19.2 sed – Stream Editor

sed reads input line by line, applies editing commands, and writes to stdout. Primarily used for substitution,
deletion, and line selection.

sed 's/old/new/' file Replace first occurrence of old with new per line.

sed 's/old/new/g' file Replace all occurrences (global).

sed 's/old/new/gi' file Case-insensitive global replace.

sed -i 's/old/new/g' file In-place edit (modifies file directly).

sed -[Link] 's/a/b/g' file In-place edit with .bak backup.

sed '3d' file Delete line 3.

sed '2,5d' file Delete lines 2 through 5.

sed '/pat/d' file Delete lines matching pattern.

sed -n '5,10p' file Print only lines 5-10.

sed -n '/pat/p' file Print only matching lines.

sed '1i Header line' file Insert 'Header line' before line 1.

sed '$a Footer line' file Append 'Footer line' after last line.

OS & Unix/Linux Notes Page 34

 sed 's/^/ /' file Indent each line by 2 spaces.

sed '/^$/d' file Delete all blank lines.

19.3 awk – Pattern Scanning & Processing

awk is a powerful text processing language. It processes input line by line, splitting each into fields ($1, $2, ...,
$NF). NR=record number, NF=number of fields.

awk '{print $1}' file Print first field of each line.

awk '{print $1,$3}' file Print fields 1 and 3.

awk '{print NR, $0}' file Print line number and whole line.

awk -F: '{print $1}' Set : as field separator (colon).

/etc/passwd

awk 'NR==5' file Print only line 5.

awk 'NR>=3 && NR<=7' file Print lines 3-7.

awk '/pattern/' file Print lines matching pattern.

awk '$3 > 100' file Print lines where field 3 > 100.

awk '{sum+=$2} END{print Sum of column 2.

sum}' f

awk 'BEGIN{FS=":"}{print BEGIN block: set field separator.

$1}' f

awk '{print NF}' file Print number of fields per line.

awk 'END{print NR}' file Print total number of lines.

19.4 sort – Sort Lines

sort file Sort lines alphabetically.

sort -r file Reverse sort.

sort -n file Numeric sort.

sort -rn file Reverse numeric sort.

sort -k2 file Sort by field 2.

sort -k2 -n file Numeric sort by field 2.

sort -t: -k3 -n file Sort by field 3 using : as delimiter.

sort -u file Sort and remove duplicates (unique).

sort -f file Ignore case (fold).

19.5 uniq – Remove Duplicate Lines

uniq file Remove adjacent duplicate lines (sort first).

uniq -c file Count occurrences of each line.

uniq -d file Print only duplicate lines.

OS & Unix/Linux Notes Page 35

 uniq -u file Print only unique (non-duplicate) lines.

uniq -i file Case-insensitive comparison.

sort file | uniq -c | sort Top frequency word/line count.

-rn

19.6 cut – Extract Columns

cut -d: -f1 /etc/passwd Extract field 1 using : delimiter.

cut -d, -f1,3 [Link] Extract fields 1 and 3 from CSV.

cut -c1-10 file Extract characters 1 through 10.

cut -c5- file Extract from character 5 to end.

cut -f2-4 file Extract fields 2-4 (default tab delimiter).

19.7 tr – Translate Characters

tr 'a-z' 'A-Z' < file Convert lowercase to uppercase.

tr -d '\n' < file Delete all newlines.

tr -s ' ' < file Squeeze multiple spaces to one.

tr -d '0-9' < file Delete all digits.

tr '[:lower:]' '[:upper:]' POSIX class uppercase conversion.

< f

OS & Unix/Linux Notes Page 36

 CHAPTER 20: Comparing Files in Unix
20.1 diff – Compare Files Line by Line
diff compares two files and shows which lines differ. Output uses < for lines in file1 only and > for lines in file2
only.

diff file1 file2 Show differences between two files.

diff -u file1 file2 Unified format (patch-friendly). ±3 context lines.

diff -c file1 file2 Context format (older format).

diff -y file1 file2 Side-by-side comparison.

diff -i file1 file2 Ignore case differences.

diff -b file1 file2 Ignore whitespace differences.

diff -B file1 file2 Ignore blank line differences.

diff -r dir1 dir2 Recursively compare directories.

diff -q file1 file2 Only report whether files differ (quiet).

Reading diff output:
3c3 # line 3 in file1 Changed to line 3 in file2
< old line # line from file1
---
> new line # line from file2

5d4 # line 5 in file1 Deleted (becomes after line 4 in file2)

7a9 # line Added after line 7 in file1 (= line 9 in file2)

20.2 cmp – Byte-by-Byte Comparison

• cmp compares files byte by byte. Used for binary files.
• If identical: no output and exit code 0.
• If different: reports first byte/line that differs.

cmp file1 file2 Report first difference (byte and line number).

cmp -l file1 file2 List all differing bytes (octal values).

cmp -s file1 file2 Silent — just set exit code (0=same, 1=differ).

20.3 comm – Compare Sorted Files

comm compares two sorted files and outputs three columns: lines only in file1, lines only in file2, lines in both.

comm file1 file2 Three-column output (sort files first).

comm -12 file1 file2 Print only common lines (suppress cols 1 and 2).

comm -23 file1 file2 Print only lines unique to file1.

comm -13 file1 file2 Print only lines unique to file2.

comm -1 file1 file2 Suppress column 1 (unique to file1).

OS & Unix/Linux Notes Page 37

 CHAPTER 21: Counting File Data – wc Command
21.1 wc – Word Count
wc file Output: lines words bytes filename.

wc -l file Count lines only.

wc -w file Count words only (whitespace-delimited).

wc -c file Count bytes.

wc -m file Count characters (may differ from bytes in multibyte).

wc -L file Length of longest line.

wc *.txt Count all txt files, show totals.

cat file | wc -l Count lines from stdin.

wc -l < file Count lines (no filename in output).

$ wc [Link]
142 1089 7342 [Link]
# 142 lines, 1089 words, 7342 bytes

$ ls -l | wc -l # count files in directory

$ grep -c 'error' log # count error lines (grep -c is faster)

OS & Unix/Linux Notes Page 38

 CHAPTER 22: Unix File Ownership & Permissions
22.1 Ownership Concepts
• Every file has an owner (UID) and a group (GID).
• Only root can change a file's owner. A user can change group to any of their groups.
• When a user creates a file, owner = that user, group = user's primary group.
• /etc/passwd — user account info. /etc/group — group membership info.
• /etc/shadow — encrypted passwords (readable only by root).
id Show current user's UID, GID, and groups.

id username Show UID, GID, groups of another user.

groups List groups current user belongs to.

who Show who is logged in.

w Show who is logged in and what they're doing.

whoami Print effective username.

last Show login history.

newgrp groupname Switch primary group for this session.

22.2 Permission Details

Permission octal values:

Octal Binary Symbolic Meaning

7 111 rwx Read, write, execute

6 110 rw- Read, write

5 101 r-x Read, execute

4 100 r-- Read only

3 011 -wx Write, execute

2 010 -w- Write only

1 001 --x Execute only

0 000 --- No permissions

22.3 Default Permissions and sudo

• sudo: Execute command as another user (default: root). Logs all actions.
• sudo cmd — run cmd as root. sudo -u alice cmd — run as alice.
• su: Switch user. su - alice — login shell as alice. su - — become root.
• visudo: Safely edit /etc/sudoers (grants sudo privileges).
# /etc/sudoers entry examples:
alice ALL=(ALL:ALL) ALL # alice can run anything as anyone
bob ALL=(ALL) NOPASSWD:/usr/bin/apt # bob can run apt without password

OS & Unix/Linux Notes Page 39

 %wheel ALL=(ALL) ALL # all users in wheel group

OS & Unix/Linux Notes Page 40

 CHAPTER 23: Unix Process Management
23.1 Processes, PIDs and Hierarchy
• Every Unix process has a PID (Process ID) and a PPID (Parent Process ID).
• init/systemd (PID 1) is ancestor of all user processes.
• Process group: related processes. Session: process groups from one login.
ps List processes for current terminal.

ps -ef All processes, full format.

ps aux All processes, BSD format.

ps -ejH Process hierarchy (forest).

pstree Visual tree of process hierarchy.

pstree -p Include PIDs in tree.

top Dynamic real-time process list.

pgrep name Get PID by process name.

pidof cmd Get all PIDs of command.

23.2 Signals
SIGHUP (1) Hangup. Terminal disconnected. Daemons reload config on SIGHUP.

SIGINT (2) Interrupt. Ctrl+C. Terminate process.

SIGQUIT (3) Quit. Ctrl+\. Terminate + core dump.

SIGKILL (9) Kill. Cannot be caught, blocked, or ignored. Force kill.

SIGTERM (15) Terminate. Polite kill — allows cleanup. Default signal of kill.

SIGSTOP (19) Stop process. Cannot be caught. Ctrl+Z sends SIGTSTP (can be caught).

SIGCONT (18) Continue stopped process.

SIGSEGV (11) Segmentation fault — invalid memory access.

SIGALRM (14) Alarm clock — from alarm() system call.

SIGCHLD (17) Child stopped or terminated. Sent to parent.

SIGUSR1 (10) User-defined signal 1. Application-specific use.

SIGUSR2 (12) User-defined signal 2.

23.3 Job Control

cmd & Run cmd in background.

Ctrl+Z Suspend foreground process (SIGTSTP).

jobs List background/suspended jobs.

OS & Unix/Linux Notes Page 41

 fg %1 Bring job 1 to foreground.

bg %1 Resume job 1 in background.

disown %1 Remove job from shell's job list (survives shell exit).

wait Wait for all background jobs to finish.

kill %1 Kill job 1 by job number.

nohup cmd & Run immune to SIGHUP. Survives logout.

OS & Unix/Linux Notes Page 42

 CHAPTER 24: Shell Programming
24.1 What is a Shell Script?
A shell script is a text file containing a sequence of shell commands. The first line specifies the interpreter:
#!/bin/bash (shebang). Make executable with chmod +x [Link], then run with ./[Link] or bash [Link].

24.2 Variables
#!/bin/bash
name="Alice" # no spaces around =
age=30
echo "Hello $name, you are $age"
echo "Hello ${name}!"

# Read-only variable
readonly PI=3.14159

# Environment variable (export to child processes)

export JAVA_HOME=/usr/lib/jvm/java-17

# Special variables
echo $0 # script name
echo $1 # first argument
echo $# # number of arguments
echo $@ # all arguments
echo $? # exit status of last command
echo $$ # PID of current script
echo $! # PID of last background process

24.3 Command Substitution & Shell Variables

# Command substitution: $() or backticks ``
today=$(date +%Y-%m-%d)
files=`ls | wc -l`
echo "Today is $today, $files files in directory"

# Arithmetic
x=10; y=3
sum=$((x + y)) # $((expr)) for arithmetic
echo $((x * y)) # 30
echo $((x % y)) # 1 (modulo)

# String operations
str="Hello World"
echo ${#str} # length: 11
echo ${str:6} # substring from 6: World
echo ${str:6:3} # 3 chars from 6: Wor
echo ${str/World/Unix} # replace: Hello Unix
echo ${str,,} # lowercase: hello world
echo ${str^^} # uppercase: HELLO WORLD

OS & Unix/Linux Notes Page 43

 24.4 Conditionals
# if-elif-else
if [ $age -ge 18 ]; then
echo "Adult"
elif [ $age -ge 13 ]; then
echo "Teenager"
else
echo "Child"
fi

# Test operators
# Numeric: -eq -ne -lt -le -gt -ge
# String: = != -z (empty) -n (not empty)
# File: -f (file) -d (dir) -e (exists) -r -w -x

# [[ ]] is bash-specific, more powerful

if [[ $name == A* ]]; then echo "starts with A"; fi
if [[ -f [Link] && -r [Link] ]]; then echo "readable file"; fi

24.5 Loops
# for loop
for i in 1 2 3 4 5; do echo $i; done
for i in {1..10}; do echo $i; done
for ((i=0; i<5; i++)); do echo $i; done
for file in *.txt; do wc -l "$file"; done

# while loop
count=0
while [ $count -lt 5 ]; do
echo "Count: $count"
((count++))
done

# until loop (opposite of while)

until [ $count -ge 10 ]; do ((count++)); done

# Loop control
break # exit loop
continue # skip to next iteration

24.6 Functions
# Define function
greet() {
local name=$1 # local variable
echo "Hello, $name!"
return 0 # return exit status
}

# Call function
greet "Alice"
greet "Bob"

OS & Unix/Linux Notes Page 44

 # Function with return value (via echo)
add() { echo $(($1 + $2)); }
result=$(add 3 4)
echo "Sum: $result"

24.7 Input/Output in Scripts

# Read user input
read -p "Enter name: " name
read -s -p "Enter password: " pass # -s = silent
read -t 10 -p "You have 10 seconds: " ans # timeout

# Redirection
echo "log" >> /var/log/[Link] # append
cmd > [Link] 2>&1 # stdout+stderr to file
cmd 2>/dev/null # discard errors

# Here document
cat << EOF
This is line 1
This is line 2
EOF

# Pipes in scripts
active_users=$(who | cut -d' ' -f1 | sort -u | wc -l)

24.8 case Statement

case $day in
Monday|Tuesday|Wednesday|Thursday|Friday)
echo "Weekday" ;;
Saturday|Sunday)
echo "Weekend" ;;
*)
echo "Unknown" ;;
esac

OS & Unix/Linux Notes Page 45

 CHAPTER 25: Vi / Vim Editor
25.1 Vi Modes
Normal Mode Default mode. Navigate, copy, delete, search. Press Esc to return here.

Insert Mode Type text. Enter with i (before cursor), a (after cursor), o (new line below).

Visual Mode Select text. Enter with v (char), V (line), Ctrl+v (block).

Command Mode Enter : then command. :wq save+quit, :q! quit without saving.

25.2 Essential Vi/Vim Commands

i Insert before cursor.

a Append after cursor.

o Open new line below and enter insert.

O Open new line above and enter insert.

Esc Return to normal mode.

:w Save file.

:q Quit (fails if unsaved changes).

:wq or :x Save and quit.

:q! Quit without saving.

:w filename Save as filename.

h j k l Move: left, down, up, right.

w / b Move word forward / backward.

0 / $ Go to line start / end.

gg / G Go to first / last line.

:n Go to line n (e.g., :42).

dd Delete (cut) current line.

ndd Delete n lines (e.g., 5dd).

yy Yank (copy) current line.

p / P Paste after / before cursor.

u Undo.

Ctrl+r Redo.

/pattern Search forward for pattern.

?pattern Search backward.

n / N Next / previous search result.

:%s/old/new/g Global substitute in file.

OS & Unix/Linux Notes Page 46

 :n,ms/o/n/g Substitute in lines n to m.

v then y Visual select then yank.

>> / << Indent / unindent line.

:set nu Show line numbers.

:set nonu Hide line numbers.

:syntax on Enable syntax highlighting.

OS & Unix/Linux Notes Page 47

 CHAPTER 26: Advanced Filtering & Text Processing
26.1 Pipes and Redirection
The pipe | connects stdout of one command to stdin of the next. This is the Unix philosophy in action: compose
simple tools into powerful pipelines.
# Common pipeline patterns
ps aux | grep nginx # find nginx processes
cat /etc/passwd | cut -d: -f1 | sort # sorted list of users
ls -la | awk '{print $9, $5}' # filename and size
history | awk '{print $2}' | sort | uniq -c | sort -rn | head
# most-used commands
find . -name '*.log' | xargs wc -l # count lines in all logs
du -sh */ | sort -rh | head -10 # top 10 largest dirs

26.2 Redirection Operators

> Redirect stdout to file (overwrite). echo hello > [Link]

>> Redirect stdout to file (append). echo line >> [Link]

< Redirect file to stdin. sort < [Link]

2> Redirect stderr. cmd 2> [Link]

2>&1 Redirect stderr to same place as stdout. cmd > [Link] 2>&1

&> Redirect both stdout and stderr. cmd &> [Link] (bash shorthand)

| Pipe stdout to next command's stdin.

tee Read stdin, write to stdout AND file. cmd | tee [Link]

/dev/null Black hole — discard unwanted output. cmd > /dev/null 2>&1

26.3 xargs – Build Command Lines from Input

find . -name '*.txt' | Cat all found txt files.
xargs cat

find . -name '*.log' | Delete all found log files.

xargs rm

echo 'a b c' | xargs -n1 One argument per line.

echo

cat [Link] | xargs -P4 Parallel wget with 4 processes.

wget

find . | xargs grep Grep across many files.

'pattern'

ls *.jpg | xargs -I{} cp {} Replace {} with each filename.

/backup/

OS & Unix/Linux Notes Page 48

 26.4 find – Find Files
find / -name '[Link]' Find by exact name.

find . -iname '*.TXT' Case-insensitive name.

find . -type f Find regular files only.

find . -type d Find directories only.

find . -type l Find symbolic links.

find . -size +100M Files larger than 100MB.

find . -mtime -7 Modified in last 7 days.

find . -mtime +30 Not modified in last 30 days.

find . -newer [Link] Newer than reference file.

find . -perm 644 Files with exact permissions 644.

find . -perm /o+w World-writable files.

find . -user alice Files owned by alice.

find . -name '*.tmp' Find and delete .tmp files.

-delete

find . -name '*.c' -exec Find and compile each .c file.

gcc {} ;

find . -name '*.log' -exec Execute once with all found files.
ls -lh {} +

26.5 Regular Expressions Quick Reference

Regex Meaning Example

. Any single character a.c matches abc, axc

* Zero or more of preceding ab* matches a, ab, abb

+ One or more (ERE) ab+ matches ab, abb

? Zero or one (ERE) ab? matches a, ab

^ Start of line ^hello matches lines starting with hello

world$ matches lines ending with

$ End of line
world

[abc] Character class [aeiou] matches any vowel

[^abc] Negated class [^0-9] matches non-digit

{n,m} Between n and m times a{2,4} matches aa, aaa, aaaa

\b Word boundary \bcat\b matches cat not catch

| Alternation (ERE) cat|dog matches cat or dog

() Group (ERE) (ab)+ matches ab, abab

OS & Unix/Linux Notes Page 49

 \d Digit (PCRE) \d{3} matches three digits

\w Word char (PCRE) \w+ matches word

\s Whitespace (PCRE) \s+ matches spaces/tabs

End of Notes — Operating Systems, Unix & Linux Complete Reference

OS & Unix/Linux Notes Page 50