File System Interface Overview
File System Interface Overview
Interface
Operating System Concepts – 10th Edition, Silberschatz, Galvin and Gagne ©2018
Chapter 10: File-System Interface
n File Concept
n Access Methods
n Directory Structure
n File-System Mounting
n File Sharing
n Protection
Operating System Concepts – 10th Edition 10-11.2 Silberschatz, Galvin and Gagne ©2018
Objectives
n To explain the function of file systems
n To describe the interfaces to file systems
n To discuss file-system design tradeoffs, including access
methods, file sharing, file locking, and directory structures
n To explore file-system protection
Operating System Concepts – 10th Edition 10-11.3 Silberschatz, Galvin and Gagne ©2018
Operating Systems – Abstraction
n Make the system easier to use.
n Abstract many of the details of the physical hardware away
l Scheduling – illusion of exclusive execution
l Virtual Memory – illusion of exclusive memory access
n Need abstractions/interfaces for
l Interacting with devices (disks, terminals, keyboards)
l Interacting with other processes
n Abstraction of Disk
l Disk, like memory is a large array of bits
l Partitioned into sectors (usually 512 bytes)
Unit of transfer to and from disk
l Non-volatile – data persists across reboots/power downs
l How do we share the disk between processes?
Protection/Security
Software
Result
Media Time
Queue
(Seek+Rot+Xfer)
(Device Driver)
n Highest Bandwidth:
l transfer large group of blocks sequentially from one track
Operating System Concepts – 10th Edition 10-11.7 Silberschatz, Galvin and Gagne ©2018
Magnetic Disk (Cont.)
n Disks: Not just slow, Complex n Bad sectors
l Seeks between cylinders are l Allocate spare sectors per track
expensive
l Block can be mapped to spare in
Random vs. sequential factory, by device controller, by
access OS
n Sequential access – platter l OS can hide bad sectors by
rotation allocating them to a special
l 80 MB/s bandwidth hidden file
l 0.16 ms latency l Physical backup programs have
to be careful
n Random access – head seeks
l Disk can be subdivided into partitions
l 0.5 MB/s bandwidth
which can be used ad raw – without
l 10 ms latency a file system, or formatted (having
n All sorts of fail structure) with a file system.
l Lots of errors possible l Entity containing file system known
as a volume. Each volume containing
l E.g., latent sector errors,
file system also tracks that file
mis-directed writes, etc.
system’s info in device directory or
l Transient vs. hard errors
Operating System Concepts – 10th Edition 10-11.8volume table of contents
Silberschatz, Galvin and Gagne ©2018
File Systems
n 3 criteria for long-term information storage:
l Should be able to store very large amount of information
l Information must survive the processes using it
l Should provide concurrent access to multiple processes
n Solution:
l Store information on disks in units called files
l Files are persistent, and only owner can explicitly delete it
l Files are managed by the OS
n File Systems: How the OS manages files!
n File systems have different types
l File system partitions have types
e.g., swap space, EXT3, FAT32, NTFS
Operating System Concepts – 10th Edition 10-11.9 Silberschatz, Galvin and Gagne ©2018
File System (Cont.)
n File System: Layer of OS that transforms block interface of disks (or
other block devices) into Files, Directories, etc.
Operating System Concepts – 10th Edition 10-11.10 Silberschatz, Galvin and Gagne ©2018
File Systems (Cont.)
n A file system provides an abstraction for storing, organizing and
accessing persistent data. It is a Layer of OS that transforms block
interface of disks (or other block devices) into Files, Directories, etc.
l File-system data is organized as logical files
Accessed via system calls
Files have names and are organized as directories
l Directory structure maintains info of all files in the volume
Name, location, size, type, …
The list of blocks containing the data
Other information such as access control list or permissions,
owner, time of access, etc?
11
Operating System Concepts – 10th Edition 10-11.11 Silberschatz, Galvin and Gagne ©2018
Translating from User to System View
File
System
n Example
l With a 512 byte sector size, a 64 MB file (MB = 1024*
1024 bytes) would have how many sectors?
21
Operating System Concepts – 10th Edition 10-11.21 Silberschatz, Galvin and Gagne ©2018
Structures That Implement a File System
On-disk structures See figure in In-memory structures
Slide 24/33 n Partition table
n Boot control block
l Info to boot the OS from the n Current mounted partitions
disk n Directory structure
l Typically, the first block of the l Information on recently
boot partition (boot block) access directories
n Partition control block n System-wide, open file table
l Info about a file system l All the currently open files
(number of blocks -- fixed size)
n Per-process, open file table
l Includes free block information
l All this process’s open files
(superblock)
n Directory Structure
n File Control Blocks
l Logical file system’s
representation of a file --
stored on disk
In UNIX, called an inode
Operating System Concepts – 10th Edition 10-11.22 Silberschatz, Galvin and Gagne ©2018
File System Design
n A file system performs four main tasks
l Block allocation and placement
Maps (potentially non-contiguous) blocks to the file
Issues similar to virtual memory, placement unique to disks
l Directory management
Maps file names to location of starting block of file
Operating System Concepts – 10th Edition 10-11.24 Silberschatz, Galvin and Gagne ©2018
Common methods of allocation
n Directories and Files needs to be allocated on disk. Directories and
files may occupy multiple data blocks on the disk.
l Not only directories/Files but also their FCB may actually be multiple
blocks on disk ( E.g., if information about data blocks require more
space than can fit in a single disk block)
n Linked:
l Use space in each disk block for a pointer to next block; indicate
end block with special value pointer (e.g., ‐1 or 0)
l FCB has pointer to first block in linked list
n Contiguous:
l Blocks for a specific file allocated together in a sequence on the
physical device
l FCB records number of blocks in file and a pointer to first block
n Indexed:
l Use separate (index) blocks of disk space for pointers to data
blocks of the file
l FCB contains pointer(s) to index
Operating System Concepts – 10th Edition 10-11.25
block(s) Silberschatz, Galvin and Gagne ©2018
Contiguous Allocation
n All blocks in a file are contiguous on the disk
n Advantages
l Easy to find file’s block. Need starting disk address and number
of blocks (length of file). Efficient for direct access.
l Performance is good for sequential reading ( need one seek only)
n Disadvantages
l File size has to be known a priori.
l File growth requires copying
l Disk becomes fragmented after deletion (external fragmentation)
l Will need periodic compaction (No special HW like MMU!!!!)
n Good for CD-ROMs where Files are never deleted
l All file sizes are known in advance
Operating System Concepts – 10th Edition 10-11.26 Silberschatz, Galvin and Gagne ©2018
Assume that disk block=1024
bytes.
What is the physical location
of logical byte 1045 in file
mail?:
you need one disk access
(reference) assuming that the
directory FCB is stored in
memory. It is located in the
21th byte within Block number
20 that is relative to the known
Operating System Concepts – 10th Edition 10-11.27 starting physical block
Silberschatz, 19.
Galvin and Gagne ©2018
Non-contiguous: 1- Linked Allocation
Each file is a linked list of
blocks. Keep a pointer to first
block of a file in the directory.
The first few bytes of each block
point to the next block of this Directory
file.
Advantages: No external File A 4
fragmentation but internal in
File B 6
the last block.
Disadvantages: 1- Random access
is slow!. 2- Reliability: If one block File A: blocks 4, 7, 2, 10, 12
pointer is corrupted, we loose file File B: Blocks 6 , 3, 11, 14
after that point
Operating System Concepts – 10th Edition 10-11.28 Silberschatz, Galvin and Gagne ©2018
Assume that disk block=1K bytes &
block address is 4 bytes. Each block
can store 1024-4=1020 bytes
What is the physical location of
logical byte 4079 in file jeep:
you need i disk access (reference)
where i = 4 assuming that the
directory FCB is stored in memory. It
is located in the last (1023th) byte
within Block number 10 that is relative
to the known starting block 9.
Operating System Concepts – 10th Edition 10-11.29 Silberschatz, Galvin and Gagne ©2018
Indexed Allocation
n In the prev. scheme, we needed to go to disk to chase pointers since
memory cannot hold all the blocks.
n Store separate blocks, per file, containing pointers to disk blocks
contained in the file
n Two types of blocks per file
l Index blocks
l Data blocks
n Positive aspects
l More efficient method for direct access (than linked)
l Also good for sequential access
n Issues
n – Overhead for pointers increases; Why?
n – Fragmentation
n What if a large file requires more space than a single index block?
l Linked index blocks
lSystem Concepts – 10th Edition
Operating Mulilevel indexing 10-11.30 Silberschatz, Galvin and Gagne ©2018
Same example as in list allocation but needs
2 disk accesses to locate byte 4079. one for
block 19 (that contains all the links for file
jeep) and then block 10. Direct access
What if we keep index 19 in memory?
Operating System Concepts – 10th Edition
Yields10-11.31
one disk access Silberschatz, Galvin and Gagne ©2018
Fragmentation Allocation Methods
n Internal: Areas within a file that are unused
n External: Blocks outside of files that are unused
Fragmentation Contiguous Linked Indexed
Internal May have to On average ½ of • On average ½ of
specify the last block goes last index block
maximum unused goes unused
extent to which
the file will grow • On average ½ of
last data block
goes unused
Operating System Concepts – 10th Edition 10-11.32 Silberschatz, Galvin and Gagne ©2018
File Allocation Table (FAT) File system
n File allocation table resides at the beginning of the volume.
n Root folder must be stored in a fixed location so that the files needed to
start the system can be correctly located.
n A volume formatted with the FAT file system is allocated in clusters.
n A partition is divided up into identically sized clusters, small blocks of
contiguous space.
l Cluster sizes vary depending on the type of FAT file system being
used and the size of the partition, typically cluster sizes lie
somewhere between 2 KB and 32 KB.
l Each file may occupy one or more of these clusters depending on its
size; thus, a file is represented by a chain of these clusters (referred
to as a singly linked list). However these clusters are not
necessarily stored adjacent to one another on the disk's surface
but are often instead fragmented throughout the Data Region
Operating System Concepts – 10th Edition 10-11.33 Silberschatz, Galvin and Gagne ©2018
File Allocation Table (FAT) File system
System Bytes Per Cluster Within Cluster limit
File Allocation Table
Operating System Concepts – 10th Edition 10-11.36 Silberschatz, Galvin and Gagne ©2018
UNIX I-node Structure
n List allocation spreads index information on disk, slowing random access
n FAT keeps
linked-list index
information in
memory but that
limits size of file
system
n I-node Idea
n inode is the UNIX FCB (file control block) i-node contains:
n Variable level indexed allocation method File attributes + 10
direct pointers+ 3
l Direct levels of indirect
Pointers contained in FCB pointers
These disk block addresses refer directly to data blocks stored on
disk
l Variable level indirect index blocks
These refer to index blocks, which store blocks of disk addresses
Operating System Concepts – 10th Edition 10-11.37 Silberschatz, Galvin and Gagne ©2018
The UNIX File-System Data Structures
Operating System Concepts – 10th Edition 10-11.38 Silberschatz, Galvin and Gagne ©2018
The UNIX File-System Data Structures(Cont.)
n Suppose that a file is 1MB and a block is 8KB. Then the file will have
128 blocks. The question is: how do we keep track of these 128 data
blocks of the file. We can do this by using index blocks (also called
indirect blocks) that contain pointers to other index and data blocks.
n In Unix each file or directory has inode that is 64 or 128 bytes and contains
l Attributes +
l 13 pointers
The first 10 pointers are to store the addresses of first 10 data blocks.
The 11th pointer points to a single indirection index block, i.e. an
index block with pointers to data blocks(points to 2^10 Data blocks)
The 12th pointer points to a double indirection index block, i.e. an
index block that points to index blocks that point to data blocks (2^20)
The 13th pointer points to a triple indirection index block. (2^30)
n If we assume that a block is 8KB and a pointer is 8Bytes, then an index
block will contain 1K pointers. Then, it can point to 1024 data blocks. The
maximum size of a file will be
8KB*(10 + 2**10 + 2**20 + 2**30), that is more than 8TByte file size.
Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 10-11.39
The UNIX File-System Data Structures(Cont.)
n Access time is different at different locations in a file. Again assume
that a block is 8KB and a pointer is 8Bytes, then an index block will
contain 1K pointers
l The first 10 data blocks are accessed with a single read (the pointers
are in main memory where the inode is brought when the file is
opened). From 0 till (8*10KB - 1)
Logical byte o or 100 or 5000 or 50239 needs single disk access.
l The next data blocks require up to two reads, one for the index
block and one for the data block. (byte addresses from 80KB till
8*1024KB -1 )
l The next data blocks require up to three reads (8MB– 8GB -1)
l The next data blocks require up to four reads (8GB-8TB)
n Clearly access time is good at the beginning of the file and gets
worse and worse as we move towards its end.
n At which index block can you find a pointer to the data record that contains
byte 100,000,000 in a file (look at the note)
Operating System Concepts – 10th Edition 10-11.40 Silberschatz, Galvin and Gagne ©2018
Block Sizes
n Larger block sizes => higher internal fragmentation.
l Not good for small files because they waste Disk space
n Smaller block sizes => Files span multiple blocks and this need
multiple seeks and rotations to read data => reduce performance.
n Thus if the allocation unit of block is too large we waste space and if
it is small we waste time
n Larger block sizes => higher disk transfer rates
n Median file size in UNIX environments ~ 1K
n Typical block sizes are of the order of 512, 1K or 2K.
Operating System Concepts – 10th Edition 10-11.41 Silberschatz, Galvin and Gagne ©2018
Directory Structure
n A directory contains zero or more entries
l One entry per file or sub-directory that resides in the directory
n Entry maps file names to location of starting block
l Each entry has file name and attributes
In windows, each entry has
– Fname , Extension , Attributes , Time , Date , Size , First Block #
In UNIX, each entry has
– Fname, i-node #
n Block number of first block of the file
Data Block
Operating System Concepts – 10th Edition 10-11.42 Silberschatz, Galvin and Gagne ©2018
Unix Directories
i-node contains:
File attributes + 10
direct pointers+ 3
levels of indirect
pointers
Operating System Concepts – 10th Edition 10-11.43 Silberschatz, Galvin and Gagne ©2018
Accessing a file block for \a\b
DOS(windows) UNIX
n Go to “\” FAT entry (in memory) Get “/” i-node from disk (usually
fixed, e.g. #2)
n Go to corresponding data Get block after block of “/” using
block(s) of “\” to find entry for its i-node till entry for “a” is
“a” found (gives its i-node #).
n Read 1st data block of “a” to Get i-node of “a” from disk
check if “b” present. Get block after block of “a” till
n Else, use the FAT entry to find entry for “b” is found (gives its i-
the next block of “a” and search node #)
again for “b”, and so on. Get i-node of “b” from disk
Eventually you will find entry for Find out whether block you
“b”. are searching for is in 1st
10 ptrs, or 1-level or 2-level
n Read the relevant block of “b”, or 3-level indirect.
by chasing the FAT entries in Based on this you can
memory. either directly get the
block, or retrieve it after
going through the levels of
indirection.
Operating System Concepts – 10th Edition 10-11.44 Silberschatz, Galvin and Gagne ©2018
Unix Directories
Operating System Concepts – 10th Edition 10-11.45 Silberschatz, Galvin and Gagne ©2018
File Deletion
Operating System Concepts – 10th Edition 10-11.46 Silberschatz, Galvin and Gagne ©2018
End of Chapter 10
Promise is promise. What is this?
Dinosaur becomes so small!!!
Operating System Concepts – 10th Edition, Silberschatz, Galvin and Gagne ©2018