UNIT-1: Evolution of Components in Systems Programming

and Related Tools.

The explanation is divided into three major parts:

1. Evolution of System Programming Components and Software Tools

o Historical evolution from primitive assembly and macro techniques to modern

compilers and integrated programming environments.

o Overview of assemblers, loaders, linkers, macros, and compilers.

o Discussion of supporting tools: text editors, interpreters, program generators,

debug monitors, and overall programming environments.

2. Compiler Concepts

o A brief overview of the compilation process.

o The idea behind incremental compilers and their benefits.

3. Assembler and Loader Mechanisms

o The problem domain that assemblers address (translating mnemonics to

machine code) and the role of the symbol table.

o Detailed explanation of various loader schemes:

 Compile‐and‐go loaders,

 General loader schemes including absolute and relocating loaders,

 Direct linkage loaders,

 Binders and overlays.

Each section is explained in depth below.

Part 1: Evolution of System Programming Components and Software Tools

1.1 Historical Evolution and Key Concepts

1.1.1 Systems Programming Components

 Assemblers:
Convert human-readable assembly language (mnemonics) into machine code. Early
 systems relied heavily on manual assembly code, but as complexity increased, higher-
level abstractions became necessary.

 Loaders:
Responsible for loading machine code into memory for execution. They resolve
addresses and link code fragments, managing memory allocation.

 Linkers:
Combine separate object code modules into a single executable, resolving external
references (symbols) among modules.

 Macros:
Provide a mechanism to define reusable code snippets. Macro assemblers allow code
expansion before translation to machine code.

 Compilers:
Translate high-level source code into machine code (or an intermediate representation)
while performing optimizations and error-checking.

1.1.2 Software Tools in the Programming Environment

 Text Editors:
The fundamental tool for writing source code. They range from simple editors (like vi or
Emacs) to modern IDEs with syntax highlighting, auto-completion, and debugging
support.

 Interpreters:
Execute code directly from source without a separate compilation phase. They are
essential for scripting languages and rapid prototyping.

 Program Generators:
Tools that automatically generate source code or code templates from specifications or
models.

 Debug Monitors (Debuggers):

Assist developers in tracking down and correcting errors by allowing breakpoints, step-
by-step execution, variable inspection, and memory examination.

 Integrated Programming Environments:

Combine text editors, build systems, debuggers, and version control systems into a single
cohesive environment to enhance productivity and code quality.

Historical Note:
The evolution from punch cards and simple text editors to modern integrated development
environments (IDEs) reflects the increasing complexity of software and the need for better
abstraction, automation, and tooling.

Part 2: Compiler Concepts

2.1 Overview of the Compilation Process

Compilation is the process of transforming high-level source code into an executable program.
The main phases include:

1. Lexical Analysis:
The compiler reads the source code and breaks it into tokens (keywords, identifiers,
literals).

2. Syntax Analysis (Parsing):

Tokens are arranged into a parse tree based on the grammar of the language.

3. Semantic Analysis:
The compiler checks for semantic errors (e.g., type checking, scope resolution) and
annotates the parse tree with type information.

4. Intermediate Code Generation:

Converts the parse tree into an intermediate representation (IR) that is easier to
optimize.

5. Optimization:
The IR is transformed to improve efficiency (e.g., dead code elimination, loop
optimization) without changing its meaning.

6. Code Generation:
The optimized IR is translated into target machine code.

7. Assembly and Linking:

The machine code is assembled into object files, and a linker combines these into a
single executable.

2.2 Incremental Compilers

Incremental compilation is a technique where only the parts of the code that have changed are
recompiled, rather than recompiling the entire code base. Benefits include:

 Reduced Compilation Time:

Faster turnaround during development.
  Improved Developer Productivity:
Immediate feedback for code changes.

 Efficient Use of Resources:

Saves computational resources by avoiding redundant compilation steps.

Real-World Example:
Modern IDEs such as Visual Studio or Eclipse use incremental compilation techniques, so editing
a single module only recompiles that module and any that depend on it.

Part 3: Assemblers and Loader Mechanisms

3.1 Assemblers: Fundamentals and the Symbol Table

3.1.1 Problem Statement Addressed by Assemblers

 Purpose:
Assemblers translate human-readable assembly language into machine code.

 Challenges:

o Mapping Mnemonics to Opcodes:

Each mnemonic (instruction) must be converted into the corresponding machine
code.

o Address Resolution:
Labels in the assembly code must be resolved to actual memory addresses.

3.1.2 Symbol Table

 Definition:
A data structure used by the assembler to store information about symbols (labels,
variable names) encountered in the source code.

 Contents:

o Symbol Name:
The label or identifier.

o Address/Location Counter:
The memory address where the symbol is defined.

o Attributes:
Data type, scope, and sometimes size.
  Usage:
The symbol table is built during the first pass of a two-pass assembler and then
referenced in the second pass to generate the correct machine code.

Example Process:

 Pass 1:
Read the assembly code, assign addresses, and populate the symbol table.

 Pass 2:
Generate machine code using the symbol table to resolve addresses.

3.2 Loader Schemes

Loaders are responsible for loading programs into memory and preparing them for execution.
Several loader schemes exist:

3.2.1 Compile-and-Go Loader

 Definition:
A simple loading method where the source code is compiled, assembled, and
immediately loaded into memory for execution in one pass.

 Use Cases:
Often used in simple, small-scale systems or educational environments.

 Advantages:
Minimal overhead and fast turnaround.

 Disadvantages:
Limited flexibility and inefficient for large projects.

3.2.2 General Loader Schemes

General loaders can be more complex and include several variants:

Absolute Loader

 Definition:
Loads a program that has been compiled for a fixed memory address.

 Process:
The object code contains absolute addresses. The loader simply copies code to the
specified memory locations.
  Limitation:
Lacks flexibility—programs must be compiled for specific memory locations.

Relocating Loader (Relocatable Loader)

 Definition:
Loads programs that are compiled as relocatable code.

 Process:
The loader adjusts the addresses in the code (relocates them) based on the actual load
location in memory.

 Advantage:
Provides flexibility as programs do not have fixed addresses.

Direct Linkage Loader

 Definition:
Integrates the loading process with linking, resolving external references on the fly.

 Process:
The loader resolves symbol references and binds them to the correct addresses before
execution.

 Use Cases:
Often used in dynamic linking scenarios.

Binders and Overlays

 Binders:

o Definition:
Tools that combine separately compiled modules into a single executable by
resolving external references.

o Static Binding vs. Dynamic Binding:

Static binding happens at load time; dynamic binding is resolved during runtime.

 Overlays:

o Definition:
A technique to manage memory limitations by loading only the required parts
(overlays) of a program at a time.
 o Mechanism:
The program is partitioned into overlays that are loaded and unloaded as
needed. This allows large programs to run on systems with limited memory.

o Example Scenario:
Early computer systems with very limited RAM used overlays to run programs
that were larger than available memory.

Final Comprehensive Summary

This detailed explanation of UNIT-1: Evolution of Components in Systems Programming covers:

Part 1: Evolution of System Programming Components and Software Tools

 Assemblers, Loaders, Linkers, Macros, Compilers:

Evolution from early, manual coding to automated, highly optimized translation and
linking processes.

 Software Tools:
Overview of text editors, interpreters, program generators, and debug monitors that
form the programming environment. These tools facilitate code development, testing,
and debugging.

Part 2: Compiler Concepts

 Compilation Process:
A multi-phase process that transforms high-level code into machine code through lexical
analysis, parsing, semantic analysis, optimization, code generation, and linking.

 Incremental Compilation:
Techniques for re-compiling only the modified parts of a program, thus speeding up
development cycles.

Part 3: Assemblers and Loader Schemes

 Assemblers:
Address the translation of assembly language to machine code. The symbol table is a
core data structure used to map labels to memory addresses.

 Loader Schemes:
Different methods by which code is loaded into memory:

o Compile-and-Go Loader: Immediate compilation and loading.

 o Absolute Loaders: For fixed-address programs.

o Relocating Loaders: Adjust code based on actual load addresses.

o Direct Linkage Loaders: Resolve external references during loading.

o Binders and Overlays: Techniques to link modules and manage memory by

loading only necessary code segments.

This comprehensive explanation integrates historical context, theoretical foundations, detailed

mechanisms, and practical considerations. It equips you with a deep understanding of how
systems programming components have evolved and how they work together to translate, link,
and load code in a modern computing environment.
UNIT-II: Theoretical Concepts of Unix Operating System
Unix is a multiuser, multitasking operating system that has evolved over decades. Its design
emphasizes simplicity, modularity, and reusability, and it has served as the foundation for many
modern operating systems. In this unit, we explore several core aspects of Unix:

1. Basic Features of an Operating System

2. File Structure and CPU Scheduling

3. Memory Management: Swapping and Demand Paging

4. File System Architecture: Blocks, Fragments, Inodes, Directory Structure

5. User-to-User Communication

Each of these topics is explained in detail below.

1. Basic Features of an Operating System

1.1 Role and Functionality

 Definition:
An operating system (OS) is a software layer that manages hardware resources and
provides common services for computer programs.

 Core Functions:

o Process Management: Creating, scheduling, and terminating processes.

o Memory Management: Allocating and deallocating memory space.

o File System Management: Organizing, storing, retrieving, and securing files.

o Device Management: Interfacing with hardware devices through drivers.

o Security and Access Control: Ensuring that resources are accessed only by
authorized users and processes.

o User Interface: Providing a command-line or graphical interface for user

interaction.

1.2 Unix Philosophy and Design Principles

  Simplicity and Modularity:
Unix follows the “do one thing and do it well” philosophy. Each tool is designed to
perform a specific task.

 Text as a Universal Interface:

Unix uses plain text for input/output, making it easy to combine simple tools into
complex workflows (using pipes and redirection).

 Small, Reusable Programs:

Programs are developed as small, modular utilities that can be combined.

 Transparency and Extensibility:

Source code is often open, and design simplicity facilitates extensions and modifications.

2. File Structure and CPU Scheduling

2.1 File Structure in Unix

Unix file systems are designed to organize data efficiently and securely. They incorporate several
key concepts:

2.1.1 File Hierarchy

 Directory Tree:
Unix uses a hierarchical file system where everything is represented as a file (including
devices).

o Root Directory: Denoted by /, it serves as the base of the directory tree.

o Subdirectories: Organized logically (e.g., /bin, /usr, /home, /var) to separate

system binaries, user files, logs, and more.

2.1.2 File Types

 Regular Files:
Contain data (text, binaries).

 Directories:
Special files that list other files.

 Special Files:
Device files, pipes, sockets that provide interfaces to hardware and interprocess
communication.

2.1.3 File Attributes and Permissions

  Attributes:
Include owner, group, timestamps (creation, modification, access), and file size.

 Permissions:
Read (r), write (w), and execute (x) permissions for user, group, and others.

 Security Model:
Uses permission bits and sometimes access control lists (ACLs) to control file access.

2.2 CPU Scheduling in Unix

Unix systems are multitasking, meaning they schedule multiple processes concurrently. CPU
scheduling is critical for efficient system performance and responsiveness.

2.2.1 Scheduling Algorithms

 Round Robin:
Processes are given a fixed time slice (quantum) and cycled in a round-robin manner.
Simple and fair for interactive systems.

 Priority-Based Scheduling:
Processes are assigned priorities based on criteria such as user vs. system processes.
Higher-priority processes may preempt lower-priority ones.

 Multilevel Queue Scheduling:

Processes are grouped into different queues based on priority or characteristics (e.g.,
interactive vs. batch). Each queue may use a different scheduling algorithm.

2.2.2 Context Switching and Preemption

 Context Switch:
When the CPU switches from one process to another, it saves the state of the current
process and loads the state of the next one.

 Preemptive Scheduling:
The operating system can interrupt a running process to schedule a higher-priority one,
ensuring responsiveness.

Example Concept:
A typical Unix system uses a mix of round-robin for interactive processes and priority scheduling
for system tasks, balancing fairness with responsiveness.

3. Memory Management: Swapping and Demand Paging

Unix implements several memory management techniques to efficiently use physical memory
and provide isolation between processes.

3.1 Swapping

 Definition:
Swapping involves moving entire processes between main memory and secondary
storage (disk) to free up RAM.

 When Used:
Particularly in older systems or when the working set of processes exceeds available
memory.

 Advantages/Disadvantages:

o Advantage:
Enables execution of processes that otherwise wouldn’t fit in memory.

o Disadvantage:
High latency due to slow disk I/O compared to RAM access.

3.2 Demand Paging

 Definition:
Instead of swapping entire processes, demand paging loads only the required pages into
memory on demand.

 Mechanism:

o Page Fault:
Occurs when a process tries to access a page not currently in memory. The OS
loads the required page from disk.

o Virtual Memory:
Each process is given a large, contiguous virtual address space that may not all
reside in physical memory.

 Advantages:

o Reduces memory usage by loading only necessary pages.

o Supports larger address spaces than physically available RAM.

 Page Replacement Algorithms:

o Least Recently Used (LRU):

Replaces the page that hasn’t been used for the longest time.
 o First-In, First-Out (FIFO):
Replaces the oldest page in memory.

o Clock Algorithm:
A circular buffer variant of FIFO with an additional “use” bit.

4. File System: Blocks, Fragments, Inodes, and Directory Structure

Unix file systems, such as the traditional Unix File System (UFS) or its descendants, are designed
for efficient storage and retrieval of files.

4.1 Blocks and Fragments

 Blocks:
The smallest unit of storage allocated to a file. Blocks are typically of a fixed size (e.g., 4
KB).

o Advantages:
Simplicity in allocation and management.

o Disadvantages:
Internal fragmentation if files are smaller than a block.

 Fragments:
To reduce wasted space, Unix file systems may allow files to be allocated in fragments
smaller than the full block size.

o Benefit:
Better space utilization for small files.

4.2 Inodes

 Definition:
An inode (index node) is a data structure that stores metadata about a file (e.g., file size,
ownership, permissions, timestamps) and pointers to the file's data blocks.

 Key Attributes Stored in an Inode:

o File Type:
Regular file, directory, symbolic link, etc.

o Permissions and Ownership:

Read, write, execute permissions; user and group IDs.
 o Timestamps:
Creation, modification, and last access times.

o Pointers:
Direct pointers to data blocks, and indirect pointers (single, double, triple) to
handle large files.

 Role in File Access:

When accessing a file, the OS uses the inode number to quickly locate its metadata and
determine where the data blocks are stored.

4.3 Directory Structure

 Hierarchical Organization:
Directories are special files that store mappings of filenames to inode numbers. The
structure is a tree with the root directory at /.

 Directory Entries:
Each entry consists of a filename and its corresponding inode number.

 Advantages of the Unix Directory Structure:

o Scalability:
Can support a large number of files organized into subdirectories.

o Ease of Navigation:
Users and programs can traverse the directory tree using standard commands.

 Linking:
Unix supports both hard links (multiple directory entries pointing to the same inode) and
symbolic links (files that point to other files by path).

5. User-to-User Communication

Unix was designed as a multiuser system, and robust mechanisms for user communication and
interprocess communication (IPC) are integral.

5.1 Interprocess Communication (IPC)

 Pipes:
Allow one process’s output to be used as input for another process, enabling chained
commands.
 o Example:
ls -l | grep ".txt" uses a pipe to filter file listings.

 Message Queues:
Facilitate asynchronous communication between processes by placing messages in a
queue.

 Shared Memory:
Allows multiple processes to access the same region of memory for fast data exchange.

 Semaphores:
Synchronize access to shared resources, ensuring that concurrent processes do not
interfere.

 Sockets:
Enable communication over a network, allowing processes on different systems to
interact.

5.2 User Communication Tools

 Mail Systems:
Unix traditionally included simple mail programs for user-to-user communication.

 Terminal Multiplexers:
Tools like screen or tmux allow users to manage multiple terminal sessions.

 Remote Login (SSH, Telnet):

Provide secure and direct access to a Unix system over a network.

 File Sharing and Collaboration:

Unix file permissions and network file systems (like NFS) allow users to share data
securely.

Example Scenario:
Two users on a Unix system can communicate by piping the output of one program into another
or by sending messages using mail utilities. In addition, shared memory segments can be used
by applications to exchange large amounts of data quickly.

Final Comprehensive Summary

This unit provided an extremely detailed exploration of the theoretical concepts underlying the
Unix Operating System:

1. Basic Features of an Operating System:

 o Overview of the core functions—process management, memory management,
file system management, device management, security, and user interface.

o Unix’s design philosophy of simplicity, modularity, and text-based interfaces.

2. File Structure and CPU Scheduling:

o Hierarchical file system structure, file types, attributes, and permission models.

o CPU scheduling algorithms used in Unix (round robin, priority scheduling,

multilevel queues) and the concept of context switching.

3. Memory Management:

o Techniques like swapping (moving entire processes) and demand paging (loading
only required pages), along with common page replacement algorithms (LRU,
FIFO, Clock).

4. File System Internals:

o Block-based storage and fragments for efficient space utilization.

o Inodes as metadata containers, detailing file attributes and block pointers.

o Hierarchical directory structure and the mechanisms for linking files (hard and
symbolic links).

5. User-to-User Communication:

o Interprocess communication (IPC) mechanisms: pipes, message queues, shared

memory, semaphores, and sockets.

o Tools that support user communication and remote access (mail systems,
terminal multiplexers, SSH).