UNIT-4

Distributed file system

Introduction:

It allows programs to access or store isolated files as they do with the local ones, allowing programmers
to access files from any network or computer.

The DFS makes it convenient to share information and files among users on a
network in a controlled and authorized way. The server allows the client users to
share files and store data just as if they are storing the information locally.
However, the servers have full control over the data, and give access control to the
clients.

FILE SERVICE ARCHITECTURE:-

1. Flat file service: A flat file service is used to perform operations on the contents of a file. The Unique
File Identifiers (UFIDs) are associated with each file in this service. For that long sequence of bits is used
to uniquely identify each file among all of the available files in the distributed system. When a request
is received by the Flat file service for the creation of a new file then it generates a new UFID and returns
it to the requester.

Flat File Service Model Operations:

Read(FileId, i, n) -> Data: Reads up to n items from a file starting at item ‘i’ and returns it in Data.

Write(FileId, i, Data): Write a sequence of Data to a file, starting at item I and extending the file if
necessary.

Create() -> FileId: Creates a new file with length 0 and assigns it a UFID.

Delete(FileId): The file is removed from the file store.

GetAttributes(FileId) -> Attr: Returns the file’s file characteristics.

SetAttributes(FileId, Attr): Sets the attributes of the file.

2. Directory Service: The directory service serves the purpose of relating file text names with their UFIDs
(Unique File Identifiers). The fetching of UFID can be made by providing the text name of the file to the
directory service by the client. The directory service provides operations for creating directories and
adding new files to existing directories.

Directory Service Model Operations:

Lookup(Dir, Name) -> FileId : Returns the relevant UFID after finding the text name in the directory.
Throws an exception if Name is not found in the directory.

AddName(Dir, Name, File): Adds(Name, File) to the directory and modifies the file’s attribute record if
Name is not in the directory. If a name already exists in the directory, an exception is thrown.

UnName(Dir, Name): If Name is in the directory, the directory entry containing Name is removed. An
exception is thrown if the Name is not found in the directory.

GetNames(Dir, Pattern) -> NameSeq: Returns all the text names that match the regular expression
Pattern in the directory.

3. Client Module: The client module executes on each computer and delivers an integrated service
(flat file and directory services) to application programs with the help of a single API. It stores
information about the network locations of flat files and directory server processes. Here, recently
used file blocks hold in a cache at the client-side, thus, resulting in improved performance.

CASE STUDY 1:- SUN NETWORK FILE SYSTEM

All implementations of NFS support the NFS protocol – a set of remote procedure calls that provide the
means for clients to perform operations on a remote file store. The NFS protocol is operating system–
independent but was originally developed for use in networks of UNIX systems, and we shall describe
the UNIX implementation the NFS protocol (version 3).The NFS server module resides in the kernel on
each computer that acts as an NFS server. Requests referring to files in a remote file system are
translated by the client module to NFS protocol operations and then passed to the NFS server module at
the computer holding the relevant file system.
The NFS client and server modules communicate using remote procedure calls. Sun’s RPC system,
described in Section 5.3.3, was developed for use in NFS. It can be configured to use either UDP or TCP,
and the NFS protocol is compatible with both. A port mapper service is included to enable clients to bind
to services in a given host by name. The RPC interface to the NFS server is open: any process can send
requests to an NFS server; if the requests are valid and they include valid user credentials, they will be
acted upon. The submission of signed user credentials can be required as an optional security feature, as
can the encryption of data for privacy and integrity.

Virtual file system • Figure 12.8 makes it clear that NFS provides access transparency: user programs
can issue file operations for local or remote files without distinction. Other distributed file systems may
be present that support UNIX system calls, and if so, they could be integrated in the same way.

The integration is achieved by a virtual file system (VFS) module, which has been added to the UNIX
kernel to distinguish between local and remote files and to translate between the UNIX-independent file
identifiers used by NFS and the internal file identifiers normally used in UNIX and other file systems. In
addition, VFS keeps track of the filesystems that are currently available both locally and remotely, and it
passes each request to the appropriate local system module (the UNIX file system, the NFS client
module or the service module for another file system).The file identifiers used in NFS are called file
handles. A file handle is opaque to clients and contains whatever information the server needs to
distinguish an individual file. In UNIX implementations of NFS, the file handle is derived from the file’s i-
node number by adding two extra fields as follows (the i-node number of a UNIX file is a number that
serves to identify and locate the file within the file system in which the file is stored):

NFS adopts the UNIX mountable filesystem as the unit of file grouping defined in the preceding
section. The filesystem identifier field is a unique number that is allocated to each filesystem
when it is created (and in the UNIX implementation is stored in the superblock of the file
system). The i-node generation number is needed because in the conventional UNIX file system
i-node numbers are reused after a file is removed. In the VFS extensions to the UNIX file
system, a generation number is stored with each file and is incremented each time the i-node
number is reused (for example, in a UNIX create system call). The client obtains the first file
handle for a remote file system when it mounts it. File handles are passed from server to client
in the results of lookup, create and mkdir operations (see Figure 12.9) and from client to server
in the argument lists of all server operations.
The virtual file system layer has one VFS structure for each mounted file system and one v-node
per open file. A VFS structure relates a remote file system to the local directory on which it is
mounted. The v-node contains an indicator to show whether a file is local or remote. If the file
is local, the v-node contains a reference to the index of the local file (an i-node in a UNIX
implementation). If the file is remote, it contains the file handle of the remote file.

Client integration • The NFS client module plays the role described for the client module in our
architectural model, supplying an interface suitable for use by conventional application
programs. But unlike our model client module, it emulates the semantics of the standard UNIX
file system primitives precisely and is integrated with the UNIX kernel. It is integrated with the
kernel and not supplied as a library for loading into client processes so that:
• user programs can access files via UNIX system calls without recompilation or
reloading;
• a single client module serves all of the user-level processes, with a shared cache
of recently used blocks (described below);

Access control and authentication • Unlike the conventional UNIX file system, the NFS server is
stateless and does not keep files open on behalf of its clients. So the server must check the
user’s identity against the file’s access permission attributes afresh on each request, to see
whether the user is permitted to access the file in the manner requested. The Sun RPC protocol
requires clients to send user authentication information (for example, the conventional UNIX
16-bit user ID and group ID) with each request and this is checked against the access permission
in the file attributes. These additional parameters are not shown in our overview of the NFS
protocol in Figure 12.9; they are supplied automatically by the RPC system. In its simplest form,
there is a security loophole in this access-control mechanism. An NFS server provides a
conventional RPC interface at a well-known port on each host and any process can behave as a
client, sending requests to the server to access or update a file. The client can modify the RPC
calls to include the user ID of any user, impersonating the user without their knowledge or
permission. This security loophole has been closed by the use of an option in the RPC protocol
for the DES encryption of the user’s authentication information. More recently, Kerberos has
been integrated with Sun NFS to provide a stronger and more comprehensive solution to the
problems of user authentication and security; we describe this below.

 Cache:
o Client side:
 cache file data and metadata by block that is read from server in local
memory
 Cache serves as a temporary buffer for writes (allow asyncronous write)
 Advantage: reduce network usage, improve performance
 Disadvantage: write lost in memory after crash (safety vs. performance
tradeoff)
o Server side:
 server can buffer the write in memory and write to disk asychronously
 Problem: write in memory can lost
 Sol:
 battery-backed memory
 commit each WRITE to stable storage before ack WRITE success
to clients
CASE STUDY -2 ANDREW FILE SYSTEM

AFS provides transparent access to remote shared files for UNIX programs running on
workstations. Access to AFS files is via the normal UNIX file primitives, enabling existing UNIX
programs to access AFS files without modification or recompilation. AFS is compatible with NFS.
AFS servers hold ‘local’ UNIX files, but the filing system in the servers is NFS-based, so files are
referenced by NFS-style file handles rather than i-node numbers, and the files may be remotely
accessed via NFS. AFS differs markedly from NFS in its design and implementation. The
differences are primarily attributable to the identification of scalability as the most important
design goal. AFS is designed to perform well with larger numbers of active users than other
distributed file systems. The key strategy for achieving scalability is the caching of whole files in
client nodes. AFS has two unusual design characteristics: Whole-file serving: The entire
contents of directories and files are transmitted to client computers by AFS servers (in AFS-3,
files larger than 64 kbytes are transferred in 64-kbyte chunks). Whole-file caching: Once a copy
of a file or a chunk has been transferred to a client computer it is stored in a cache on the local
disk. The cache contains several hundred of the files most recently used on that computer. The
cache is permanent,surviving reboots of the client computer. Local copies of files are used to
satisfy clients’ open requests in preference to remote copies whenever possible

Operation of AFS:
1. When a user process in a client computer issues an open system call for a file in the shared
file space and there is not a current copy of the file in the local cache, the server holding the file
is located and is sent a request for a copy of the file.
2. The copy is stored in the local UNIX file system in the client computer. The copy is then
opened and the resulting UNIX file descriptor is returned to the client.
3. Subsequent read, write and other operations on the file by processes in the client computer
are applied to the local copy.
4. When the process in the client issues a close system call, if the local copy has been updated
its contents are sent back to the server. The server updates the file contents and the
timestamps on the file. The copy on the client’s local disk is retained in case it is needed again
by a user-level process on the same workstation For shared files that are infrequently updated
(such as those containing the code of UNIX commands and libraries) and for files that are
normally accessed by only a single user (such as most of the files in a user’s home directory and
its subtree), locally cached copies are likely to remain valid for long periods The local cache can
be allocated a substantial proportion of the disk space on each workstation – say, 100
megabytes. This is normally sufficient for the establishment of a working set of the files used by
one user. The provision of sufficient cache storage for the establishment of a working set
ensures that files in regular use on agiven workstation are normally retained in the cache until
they are needed again.
• The design strategy is based on some assumptions about average and maximum file size and
locality of reference to files in UNIX systems. These assumptions are derived from observations
of typical UNIX workloads in academic and other environments [Satyanarayanan 1981,
Ousterhout et al. 1985, Floyd 1986]. The most important observations are:
– Files are small; most are less than 10 kilobytes in size.
– Read operations on files are much more common than writes (about six times more
common).
– Sequential access is common, and random access is rare.
– Most files are read and written by only one user. When a file is shared, it is usually only one
user who modifies it.
– Files are referenced in bursts. If a file has been referenced recently, there is a high probability
that it will be referenced again in the near future.

AFS is implemented as two software components that exist as UNIX processes called Vice and
Venus. Figure 12.11 shows the distribution of Vice and Venus processes. Vice is the name given
to the server software that runs as a user-level UNIX process in each server computer, and
Venus is a user-level process that runs in each client computer and corresponds to the client
module in our abstract model.
The files available to user processes running on workstations are either local or shared. Local
files are handled as normal UNIX files. They are stored on a workstation’s disk and are available
only to local user processes. Shared files are stored on servers, and copies of them are cached
on the local disks of workstations. The name space seen by user.

One of the file partitions on the local disk of each workstation is used as a cache, holding the
cached copies of files from the shared space. Venus manages the cache, removing the least
recently used files when a new file is acquired from a server to make the required space if the
partition is full. The workstation cache is usually large enough to accommodate several hundred
average-sized files, rendering the workstation largely
independent of the Vice servers once a working set of the current user’s files and frequently
used system files has been cached.
AFS resembles the abstract file service model described in Section 12.2 in these respects:
• A flat file service is implemented by the Vice servers, and the hierarchic directory structure
required by UNIX user programs is implemented by the set of Venus processes in the
workstations.
• Each file and directory in the shared file space is identified by a unique, 96-bit file identifier
(fid) similar to a UFID. The Venus processes translate the pathnames issued by clients to fids.
Files are grouped into volumes for ease of location and movement. Volumes are generally
smaller than the UNIX filesystems, which are the unit of file grouping in NFS.

For example, each user’s personal files are generally located in a separate volume. Other
volumes are allocated for system binaries, documentation and library code. The representation

of fids includes the volume number for the volume containing the file (cf. the file group
identifier in UFIDs), an NFS file handle identifying the file within the volume (cf. the file number
in UFIDs) and a uniquifier to ensure that file identifiers are not reused: User programs use
conventional UNIX pathnames to refer to files, but AFS uses fids in the communication between
the Venus and Vice processes. The Vice servers accept requests only in terms of fids. Venus
translates the pathnames supplied by clients into fids using a step-by-step lookup to obtain the
information from the file directories held in the Vice servers.
Figure 12.14 describes the actions taken by Vice, Venus and the UNIX kernel when a user
process issues each of the system calls mentioned in our outline scenario above. The callback
promise mentioned here is a mechanism for ensuring that cached copies of files are updated
when another client closes the same file after updating it
Cache consistency
When Vice supplies a copy of a file to a Venus process it also provides a callback promise – a
token issued by the Vice server that is the custodian of the file, guaranteeing that it will notify
the Venus process when any other client modifies the file. Callback promises are stored with
the cached files on the workstation disks and have two states: valid or cancelled. When a server
performs a request to update a file it notifies all of the Venus processes to which it has issued
callback promises by sending a callback to each – a callback is a remote procedure call from a
server to a Venus process.

When the Venus process receives a callback, it sets the callback promise token for the relevant
file to
cancelled.
Whenever Venus handles an open on behalf of a client, it checks the cache. If the required file is
found in the cache, then its token is checked. If its value is cancelled, then a fresh copy of the
file must be fetched from the Vice server, but if the token is valid, then the cached copy can be
opened and used without reference to Vice.

When a workstation is restarted after a failure or a shutdown, Venus aims to retain as many as
possible of the cached files on the local disk, but it cannot assume that the callback promise
tokens are correct, since some callbacks may have been missed. Before the first use of each
cached file or directory after a restart, Venus therefore generates a cache validation request
containing the file modification timestamp to the server that is the custodian of the file. If the
timestamp is current, the server responds with valid and the token is reinstated. If the
timestamp shows that the file is out of date, then the server responds with cancelled and the
token is set to cancelled. Callbacks must be renewed before an open if a time T (typically on the
order of a few minutes) has elapsed since the file was cached without communication from the
server. This is to deal with possible communication failures, which can result in the loss of
callback messages.

This callback-based mechanism for maintaining cache consistency was adopted as offering the
most scalable approach, following the evaluation in the prototype (AFS-1) of a timestamp-
based mechanism similar to that used in NFS. In AFS-1, a Venus process holding a cached copy
of a file interrogates the Vice process on each open to determine whether the timestamp on
the local copy agrees with that on the server. The callback based approach is more scalable
because it results in communication between client and server and activity in the server only
when the file has been updated, whereas the timestamp approach results in a client-server
interaction on each open, even when there is a valid local copy. Since the majority of files are
not accessed concurrently, and read operations predominate over writes in most applications,
the callback mechanism results in a dramatic reduction in the number of client-server
interactions.

The callback mechanism used in AFS-2 and later versions of AFS requires Vice servers to
maintain some state on behalf of their Venus clients, unlike AFS-1, NFS and our file service
model. The client-dependent state required consists of a list of the Venus processes to which
callback promises have been issued for each file. These callback lists must be retained over
server failures – they are held on the server disks and are updated using atomic operations.
Distributed Shared Memory

DSM is a mechanism that manages memory across multiple nodes and makes inter-process
communications transparent to end-users. The applications will think that they are running on
shared memory. DSM is a mechanism of allowing user processes to access shared data without
using inter-process communications. In DSM every node has its own memory and provides
memory read and write services and it provides consistency protocols. The distributed shared
memory (DSM) implements the shared memory model in distributed systems but it doesn’t
have physical shared memory. All the nodes share the virtual address space provided by the
shared memory model.

Design and implantation issues :-

1. Granularity: Granularity refers to the block size of a DSM system. Granularity refers to the
unit of sharing and the unit of data moving across the network when a network block
shortcoming then we can utilize the estimation of the block size as words/phrases. The block
size might be different for the various networks.

2. Structure of shared memory space: Structure refers to the design of the shared data in
the memory. The structure of the shared memory space of a DSM system is regularly
dependent on the sort of applications that the DSM system is intended to support.

3. Memory coherence and access synchronization: In the DSM system the shared data things
ought to be accessible by different nodes simultaneously in the network. The fundamental issue
in this system is data irregularity. The data irregularity might be raised by the synchronous
access. To solve this problem in the DSM system we need to utilize some synchronization
primitives, semaphores, event count, and so on.

4. Data location and access: To share the data in the DSM system it ought to be possible to
locate and retrieve the data as accessed by clients or processors. Therefore the DSM system
must implement some form of data block finding system to serve network data to meet the
requirement of the memory coherence semantics being utilized.

5. Replacement strategy: In the local memory of the node is full, a cache miss at the node
implies not just a get of the gotten to information block from a remote node but also a
replacement. A data block of the local memory should be replaced by the new data block.
Accordingly, a position substitution methodology is additionally vital in the design of a DSM
system.

6. Thrashing: In a DSM system data blocks move between nodes on demand. In this way on the
off chance that 2 nodes complete for write access to the single data item. The data relating
data block might be moved to back and forth at such a high rate that no genuine work can get
gone. The DSM system should utilize an approach to keep away from a situation generally
known as thrashing.

7. Heterogeneity: The DSM system worked for homogeneous systems and need not address the
heterogeneity issue. In any case, assuming the underlined system environment is
heterogeneous, the DSM system should be designed to deal with heterogeneous, so it works
appropriately with machines having different architectures.

Consistency Model:-

Strict Consistency Model: "The strict consistency model is the strongest form of memory
coherence, having the most stringent consistency requirements. A shared-memory system is
said to support the strict consistency model if the value returned by a read operation on a
memory address is always the same as the value written by the most recent write operation to
that address, irrespective of the locations of the processes performing the read and write
operations. That is, all writes instantaneously become visible to all processes."

Sequential Consistency Model: "The sequential consistency model was proposed by Lamport ...
. A shared-memory system is said to support the sequential consistency model if all processes
see the same order of all memory access operations on the shared memory. The exact order in
which the memory access operations are interleaved does not matter. ... If one process sees
one of the orderings of ... three operations and another process sees a different one, the
memory is not a sequentially consistent memory."

Casual Consistency Model: "The causal consistency model ... relaxes the requirement of the
sequential model for better concurrency. Unlike the sequential consistency model, in the causal
consistency model, all processes see only those memory reference operations in the same
(correct) order that are potentially causally related. Memory reference operations that are not
potentially causally related may be seen by different processes in different orders.

FIFO Consistency Model: For FIFO consistency, "Writes done by a single process are seen by all
other processes in the order in which they were issued, but writes from different processes
may be seen in a different order by different processes.
"FIFO consistency is called PRAM consistency in the case of distributed shared memory
systems."

Pipelined Random-Access Memory (PRAM) Consistency Model: "The pipelined random-access

memory (PRAM) consistency model ... provides a weaker consistency semantics than the (first
three) consistency models described so far. It only ensures that all write operations performed
by a single process are seen by all other processes in the order in which they were performed
as if all the write operations performed by a single process are in a pipeline. Write operations
performed by different processes may be seen by different processes in different orders."

Weak Consistency Model: "Synchronization accesses (accesses required to perform

synchronization operations) are sequentially consistent. Before a synchronization access can be
performed, all previous regular data accesses must be completed. Before a regular data access
can be performed, all previous synchronization accesses must be completed. This essentially
leaves the problem of consistency up to the programmer. The memory will only be consistent
immediately after a synchronization operation.

Release Consistency Model: "Release consistency is essentially the same as weak consistency,
but synchronization accesses must only be processor consistent with respect to each other.
Synchronization operations are broken down into acquire and release operations. All pending
acquires (e.g., a lock operation) must be done before a release (e.g., an unlock operation) is
done. Local dependencies within the same processor must still be respected.
"Release consistency is a further relaxation of weak consistency without a significant loss of
coherence.

Entry Consistency Model: "Like ... variants of release consistency, it requires the programmer
(or compiler) to use acquire and release at the start and end of each critical section,
respectively. However, unlike release consistency, entry consistency requires each ordinary
shared data item to be associated with some synchronization variable, such as a lock or barrier.
If it is desired that elements of an array be accessed independently in parallel, then different
array elements must be associated with different locks. When an acquire is done on a
synchronization variable, only those data guarded by that synchronization variable are made
consistent.

Processor Consistency Model: "Writes issued by a processor are observed in the same order in
which they were issued. However, the order in which writes from two processors occur, as
observed by themselves or a third processor, need not be identical. That is, two simultaneous
reads of the same location from different processors may yield different results

General Consistency Model: "A system supports general consistency if all the copies of a
memory location eventually contain the same data when all the writes issued by every
processorhavecompleted."