CC - Module-1 Question With Answers
CC - Module-1 Question With Answers
1. Describe the evolution of parallel, distributed, and cloud computing over the
past 30 years with labeled diagram
The general computing trend is to leverage shared web resources and massive amounts of data
over the Internet. Figure illustrates the evolution of HPC and HTC systems. On the HPC
side,supercomputers (massively parallel processors or MPPs) are gradually replaced by clusters of
cooperative computers out of a desire to share computing resources.
The cluster is often a collection of homogeneous compute nodes that are physically connected in
close range to one another. On the HTC side, peer-to-peer (P2P) networks are formed for
distributed file sharing and content delivery applications.
A P2P system is built over many client machines. Peer machines are globally distributed in nature.
P2P, cloud computing, and web service platforms are more focused on HTC applications than on
HPC applications.
Clustering and P2P technologies lead to the development of computational grids or data grids.
High-Throughput Computing
The development of market-oriented high-end computing systems is undergoing a strategic change
from an HPC paradigm to an HTC paradigm. This HTC paradigm pays more attention to high-flux
computing. The main application for high-flux computing is in Internet searches and web services
by millions or more users simultaneously. The performance goal thus shifts to measure high
throughput or the number of tasks completed per unit of time. HTC technology needs to not only
improve in terms of batch processing speed, but also address the acute problems of cost, energy
savings, security, and reliability at many data and enterprise computing centers
Three New Computing Paradigms : with the introduction of SOA, Web 2.0 services become
available. Advances in virtualization make it possible to see the growth of Internet clouds as a new
computing paradigm.
The maturity of radio-frequency identification (RFID), Global Positioning System (GPS), and sensor
technologies has triggered the development of the Internet of Things (IoT).
• Centralized computing This is a computing paradigm by which all computer resources are
centralized in one physical system. All resources (processors, memory, and storage) are fully
Degrees of Parallelism
In this scenario, bit-level parallelism (BLP) converts bit-serial processing to word-level processing
gradually. Over the years, users graduated from 4-bit microprocessors to 8-,16-, 32-, and 64-bit
CPUs. This led us to the next wave of improvement, known as instruction-level parallelism (ILP), in
which the processor executes multiple instructions simultaneously rather than only one instruction
at a time. For the past 30 years, we have practiced ILP through pipelining, superscalar computing,
VLIW (very long instruction word) architectures, and multithreading. ILP requires branch
prediction, dynamic scheduling, speculation, and compiler support to work efficiently.
Data-level parallelism (DLP) was made popular through SIMD (single instruction, multiple data)
and vector machines using vector or array types of instructions. DLP requires even more hardware
support and compiler assistance to work properly. Ever since the introduction of multicore
processors and chip multiprocessors (CMPs), we have been exploring task-level parallelism (TLP).
A modern processor explores all of the aforementioned parallelism types.
Innovative Applications
Both HPC and HTC systems desire transparency in many application aspects. For example, data
access, resource allocation, process location, concurrency in execution, job replication, and failure
recovery should be made transparent to both users and system management
First, they are all ubiquitous in daily life. Reliability and scalability are two major design
objectives in these computing models.
Second, they are aimed at autonomic operations that can be self-organized to support
dynamic discovery.
Finally, these paradigms are composable with QoS and SLAs (service-level agreements).
The expectations rise sharply from the trigger period to a high peak of inflated expectations.
Through a short period of disillusionment, the expectation may drop to a valley and then
increase steadily over a long enlightenment period to a plateau of productivity.
The number of years for an emerging technology to reach a certain stage is marked by
The hollow circles indicate technologies that will reach mainstream adoption in two years.
The gray circles represent technologies that will reach mainstream adoption in two to five
years.
The solid circles represent those that require five to 10 years to reach mainstream adoption,
and the triangles denote those that require more than 10 years. T
The crossed circles represent technologies that will become obsolete before they reach the
plateau.
The hype cycle in Figure shows the technology status as of August 2010.
The IoT refers to the networked interconnection of everyday objects, tools, devices, or
computers. One can view the IoT as a wireless network of sensors that interconnect all
things in our daily life. These things can be large or small and they vary with respect to time
and place. The idea is to tag every object using RFID or a related sensor or electronic
technology such as GPS.
With the introduction of the IPv6 protocol, 2128 IP addresses are available to distinguish all
the objects on Earth, including all computers and pervasive devices.
Cyber-Physical Systems
A CPS merges the “3C” technologies of computation, communication ,and control into an
intelligent closed feedback system between the physical world and the information world, a
concept which is actively explored in the United States.
Both multi-core CPU and many-core GPU processors can handle multiple instruction threads at
different magnitudes today.
Figure shows the architecture of a typical multicore processor. Each core is essentially a processor
with its own private cache (L1 cache).
Multiple cores are housed in the same chip with an L2 cache that is shared by all cores. In
the future, multiple CMPs could be built on the same CPU chip with even the L3 cache on the
chip. Multicore and multithreaded CPUs are equipped with many high-end processors,
including the Intel i7, Xeon, AMD Opteron, Sun Niagara, IBM Power 6, and X cell processors.
The superscalar processor is single-threaded with four functional units. Each of the three
multithreaded processors is four-way multithreaded over four functional data paths.
In the dual-core processor, assume two processing cores, each a single-threaded two-way
superscalar processor.
Instructions from different threads are distinguished by specific shading patterns for
instructions from five independent threads.
Typical instruction scheduling patterns are shown here. Only instructions from the same
thread are executed in a superscalar processor.
Fine-grain multithreading switches the execution of instructions from different threads per
cycle.
Course-grain multithreading executes many instructions from the same thread for quite a
few cycles before switching to another thread.
The SMT allows simultaneous scheduling of instructions from different threads in the same
cycle.
The Internet of Things (IoT) refers to a network of physical devices embedded with sensors,
software, and connectivity, allowing them to collect and exchange data over the internet.
Sensor-based data collection
Real-time monitoring
Device connectivity and communication
Automation and intelligent decision-making
Cluster designers desire a cluster operating system or some middleware to support SSI at various
levels, including the sharing of CPUs, memory, and I/O across all cluster nodes. An SSI is an illusion
created by software or hardware that presents a collection of resources as one integrated, powerful
resource. SSI makes the cluster appear like a single machine to the user. A cluster with multiple
system images is nothing but a collection of independent computers.
Special cluster middleware supports are needed to create SSI or high availability (HA). Both
sequential and parallel applications can run on the cluster, and special parallel environments are
needed to facilitate use of the cluster resources. For example, distributed memory has multiple
images. Users may want all distributed memory to be shared by all servers by forming distributed
shared memory (DSM).
Like an electric utility power grid, a computing grid offers an infrastructure that couples
computers, software/middleware, special instruments, and people and sensors together. The grid
is often constructed across LAN, WAN, or Internet backbone networks at a regional, national, or
global scale.
Enterprises or organizations present grids as integrated computing resources. They can also be
viewed as virtual platforms to support virtual organizations. The computers used in a grid are
primarily workstations, servers, clusters, and supercomputers. Personal computers, laptops, and
PDAs can be used as access devices to a grid system.
Figure shows an example computational grid built over multiple resource sites owned by
different organizations. The resource sites offer complementary computing resources, including
workstations, large servers, a mesh of processors, and Linux clusters to satisfy a chain of
computational needs. The grid is built across various IP broadband networks including LANs and
WANs already used by enterprises or organizations over the Internet. The grid is presented to
users as an integrated resource pool as shown in the upper half of the figure.
Figure summarizes various attack types and their potential damage to users. As the figure shows,
information leaks lead to a loss of confidentiality.
Loss of data integrity may be caused by user alteration, Trojan horses, and service spoofing attacks.
A denial of service (DoS) results in a loss of system operation and Internet connections.
Lack of authentication or authorization leads to attackers’ illegitimate use of computing resources.
Open resources such as data centers, P2P networks, and grid and cloud infrastructures could
become the next targets. Users need to protect clusters, grids, clouds, and P2P systems.
Otherwise, users should not use or trust them for outsourced work.
Malicious intrusions to these systems may destroy valuable hosts, as well as network and storage
resources. Internet anomalies found in routers, gateways, and distributed hosts may hinder the
acceptance of these public-resource computing services.
Security Responsibilities
Three security requirements are often considered: confidentiality, integrity, and availability for
most Internet service providers and cloud users.
In the order of SaaS, PaaS, and IaaS, the providers gradually release the responsibility of security
control to the cloud users.
Copyright Protection
Collusive piracy is the main source of intellectual property violations within the boundary of a P2P
network. Paid clients (colluders) may illegally share copyrighted content files with unpaid clients
(pirates). Online piracy has hindered the use of open P2P networks for commercial content
delivery. One can develop a proactive content poisoning scheme to stop colluders and pirates from
alleged copyright Infringements in P2P file sharing. Pirates are detected in a timely manner with
identity-based signatures and time stamped tokens. This scheme stops collusive piracy from
occurring without hurting legitimate P2P clients.
System Defense Technologies
This is the primary programming standard used to develop parallel and concurrent
programs to run on a distributed system.
This is a web programming model for scalable data processing on large clusters over large
data [Link] model is applied mainly in web-scale search and cloud computing applications.
The user specifies a Map function to generate a set of intermediate key/value pairs. Then the
user applies a Reduce function to merge all intermediate values with the same intermediate
key.
MapReduce is highly scalable to explore high degrees of parallelism at different job levels. A
typical MapReduce computation process can handle terabytes of data on tens of thousands
or more client machines.
Hadoop offers a software platform that was originally developed by a Yahoo! group.
The package enables users to write and run applications over vast amounts of distributed
data.
Users can easily scale Hadoop to store and process petabytes of data in the web space.
Hadoop is economical in that it comes with an open source version of MapReduce that
minimizes overhead in task spawning and massive data communication. It is efficient, as it
processes data with a high degree of parallelism across a large number of commodity nodes,
and it is reliable in that it automatically keeps multiple data copies to facilitate
redeployment of computing tasks upon unexpected system failures.
Internet clouds offer four deployment modes: private, public, managed, and hybrid . These modes
demand different levels of security implications. The different SLAs imply that the security
responsibility is shared among all the cloud providers, the cloud resource consumers, and the third
party cloud-enabled software providers. Advantages of cloud computing have been advocated by
many IT experts, industry leaders, and computer science researchers.
10. Define and analyze the impact of three modern computing paradigms: Service-
Oriented Architecture (SOA), cloud computing and Internet of Things (IOT)
SOA applies to building grids, clouds, grids of clouds, clouds of grids, clouds of clouds (also
known as interclouds), and systems of systems in general. A large number of sensors
provide data-collection services, denoted in the figure as SS (sensor service).
A sensor can be a ZigBee device, a Bluetooth device, a WiFi access point, a personal
computer, a GPA, or a wireless phone, among other things. Raw data is collected by sensor
services. All the SS devices interact with large or small computers, many forms of grids,
databases, the compute cloud, the storage cloud, the filter cloud, the discovery cloud, and so
on.
Filter services (are used to eliminate unwanted raw data, in order to respond to specific
requests from the web, the grid, or web services.
A collection of filter services forms a filter cloud. SOA aims to search for, or sort out, the
useful data from the massive amounts of raw data items. Processing this data will generate
useful information, and subsequently, the knowledge for our daily use.
Finally, we make intelligent decisions based on both biological and machine wisdom. For
raw data collected by a large number of sensors to be transformed into useful information
or knowledge, the data stream may go through a sequence of compute, storage, filter, and
discovery clouds. Finally, the inter-service messages converge at the portal, which is
accessed by all users.
Application Layer
Until now, most user applications in science, business, engineering, and financial areas tend to
increase a system’s speed or quality. By introducing energy-aware applications, the challenge is to
design sophisticated multilevel and multi-domain energy management applications without
hurting
Middleware Layer
The middleware layer acts as a bridge between the application layer and the resource layer. This
layer provides resource broker, communication service, task analyzer, task scheduler, security
access, reliability control, and information service capabilities. It is also responsible for applying
energy-efficient techniques, particularly in task scheduling.
Resource Layer
The resource layer consists of a wide range of resources including computing nodes and storage
units. This layer generally interacts with hardware devices and the operating system; therefore, it
is responsible for controlling all distributed resources in distributed computing systems. In the
Network Layer
Routing and transferring packets and enabling network services to the resource layer are the main
responsibility of the network layer in distributed computing systems. The major challenge to build
energy-efficient networks is, again, determining how to measure, predict, and create a balance
between energy consumption and performance. Two major challenges to designing energy-efficient
networks are:
• The models should represent the networks comprehensively as they should give a full
understanding of interactions among time, space, and energy.
• New, energy-efficient routing algorithms need to be developed. New, energy-efficient protocols
should be developed against network attacks.
DVFS Method for Energy Efficiency
The DVFS method enables the exploitation of the slack time (idle time) typically incurred by
intertask relationship. Specifically, the slack time associated with a task is utilized to execute the
task in a lower voltage frequency. The relationship between energy and voltage frequency in CMOS
circuits is related by:
where v, Ceff, K, and vt are the voltage, circuit switching capacity, a technology dependent factor,
and threshold voltage, respectively, and the parameter t is the execution time of the task under
clock frequency f. By reducing voltage and frequency, the device’s energy consumption can also be
reduced.