THAMIRABHARANI ENGINEERING COLLEGE

(Approved by AICTE, New Delhi and Affiliated to Anna University, Chennai)

Chathirampudukulam, Chidambaranagar - Vepemkulam Road.
Thatchanallur, Tirunelveli 627 358, Tamil Nadu.

Name of the Faculty Member DAVID D G

Department CSE
Course Code CS3352
Name of the Course FOUNDATIONS OF DATA SCIENCE
Year/Section/Semester II/III
Department of the Course CSE
Handled

QUESTION BANK
 UNITOF
CS3352 - FOUNDATION – I DATA SCIENCE
INTRODUCTION
Question Bank
UNIT I - INTRODUCTION
Data Science: Benefits and uses – facets of data - Data Science Process: Overview – Defining
research goals – Retrieving data – Data preparation - Exploratory Data analysis – Build the
model– presenting findings and building applications - Data Mining - Data Warehousing –
Basic Statistical descriptions of Data

PART A

1. What is data science?

Data science is the study of data. It involves developing methods of recording, storing, and
analyzing data to effectively extract useful information. The goal of data science is to gain insights
and knowledge from any type of data — both structured and unstructured.
2. Explain the role of data science in business?
Data science is the requirement of every business to make business forecasts and predictions based
on facts and figures, which are collected in the form of data and processed through data science.
Data science is also important for marketing promotions and campaigns.
3. Explain the role of data science in Medical Research?
Data science is necessary for research and analysis in health care, which makes it easier for
practitioners in both fields to understand the challenges and extract results through analysis and
insights proposed based on data.
The advanced technology of machine learning and artificial intelligence has been infused in the
field of medical science to conduct biomedical and genetic researches more easily.
4. Explain the role of data science in health care?
The clinical history of a patient and the medicinal treatment given is computerized and kept as a
digitalized record, which makes it easier for the practitioner and doctors to detect the complex
diseases at an early stage and understand its complexity.
[Link] the role of data science in education?
Online personality tests and career counseling suggest students and individuals select a career path
based on their choices. This is done with the help of data science, which has designed algorithms
and predictive rationalities that conclude with the provided set of choices.
6. Explain the role of data science in social media?
Most of the data of consumer behavior, choices, and preferences are being collected through
online platforms, which help the business grow. The systems are smart enough to predict and
analyze a user’s mood, behavior, and opinions towards a specific product, incident, or event with
the help of texts, feedbacks, thoughts, views, reactions, and suggestions.
7. Explain the role of data science in Technology?
The new technology emerging around us to ease life and alter our lifestyle is all due to data
science. Some applications collect sensory data to track our movements and activities and also
provide knowledge and suggestions concerning our health, such as blood pressure and heart rate.
Data science is also being implemented to advance security and safety technology, which is the
most important issue nowadays.
8. Explain the role of data science in financial institution?
Financial institutions use data science to predict stock markets, determine the risk of lending
money, and learn how to attract new clients for their services.

TEC/CSE/CS3352-FDS/DGD
 9. Explain the main types/ categories of data?
 Structured
 Unstructured
 Natural language
 Machine-generated
 Graph-based
 Audio, video, and images
 Streaming
10. Explain natural language? Is natural language is instructed data
Yes, Natural language is a special type of unstructured data; it’s challenging to process because it
requires knowledge of specific data science techniques and linguistics. The natural language
processing is used in entity recognition, topic recognition, summarization, text completion, and
sentiment analysis. Example: Predictive text, Search results.
11. What is Machine-generated data with example?
Machine-generated data is information that’s automatically created by a computer, process,
application, or other machine without human intervention. Machine-generated data is becoming a
major data resource and will continue to do [Link] analysis of machine data relies on highly
scalable tools, due to its high volume and speed.
Examples of machine data are web server logs, call detail records, network event logs, and
telemetry.
12. What is Graph-based or network data?
Graph or network data is, in short, data that focuses on the relationship or adjacency of objects.
The graph structures use nodes, edges, and properties to represent and store graphical data. Graph-
baseddata is a natural way to represent social networks.
13. List the name of the steps involved in data science processing?
1. SETTING THE RESEARCH GOAL
2. RETRIEVING DATA
3. DATA PREPARATION
4. DATA EXPLORATION
5. DATA MODELLING
6. PRESENTATION AND AUTOMATION.
14. What are OUTLIERS?
An outlier is an observation that seems to be distant from other observations or, more specifically,
one observation that follows a different logic or generative process than the other observations.
15. What are the different ways of combining data?
The two operations that are performed in various data types to combine them are as follows:
i. Joining- enriching an observation from one table with information from another table.
ii. Appending or stacking- adding the observations of one table to those of another table.
16. What is big data?
Big data is a blanket term for any collection of data sets so large or complex that it becomes
difficult to process using traditional data management techniques. They are characterized by the
four Vs: velocity, variety, volume, and veracity.
17. What is data Cleansing?
Data cleansing is a sub process of the data science process that focuses on removing errors in your
data so your data becomes a true and consistent representation of the processes.
18. Define the term setting the research goal (in data science processing).
The term setting the research goal mean Defining what, why, and how of your projection in a
project charter.

TEC/CSE/CS3352-FDS/DGD
19. Explain the term Retrieving data.
The term retrieving data means Finding and getting access to data needed in your project. This
data is either found within the company or retrieved from a third party.
20. What is data preparation and process?
It is the process of Checking and remediating data errors, enriching the data with data from other
data sources, and transforming it into a suitable format for your models.
21. Define data exploration process?
Data exploration process of Diving deeper into your data using descriptive statistics and visual
techniques.
22. What is data modelling?
It is the process of Building a model is an iterative process that involves selecting the variables for
the model, executing the model, and model diagnostics. It is generally Using machine learning and
statistical techniques to achieve our project goal.
23. Explain Presentation and automation process?
Finally, present the results to the business. These results can take many forms, ranging from
presentations to research reports. Presenting the results to the stakeholders and industrializing the
analysis process for repetitive reuse and integration with other tools.
24. What are the types of database?
 Column databases
 Document stores
 Streaming data
 Key-value stores
 SQL on Hadoop
 New SQL
 Graph databases.
26. What is a column database?
In this method Data is stored in columns, which allows algorithms to perform much faster queries.
Newer technologies use cell-wise storage. Table-like structures are still important.
27. Explain document stores?
Document stores no longer use tables, but store every observation in a document. This allows for a
much more flexible data scheme.
28. What is Streaming data?
Data is collected, transformed, and aggregated not in batches but in real time. Although we’ve
categorized it here as a database to help you in tool selection, it’s more a particular type of
problem that drove creation of technologies such as Storm.
29. Define Key-value stores?
When Data isn’t stored in a table; rather you assign a key for every value, such as
[Link].2015: 20000. This scales well but places almost all the implementation on the
developer.
30. Explain SQL on Hadoop?
Batch queries on Hadoop are in a SQL-like language that uses the map-reduce framework in the
background
31. What is New SQL?
This class combines the scalability of NoSQL databases with the advantages of relational
databases. They all have a SQL interface and a relational data model.
32. Explain Graph databases?
Not every problem is best stored in a table. Particular problems are more naturally translated into
graph theory and stored in graph databases. A classic example of this is a social network.

TEC/CSE/CS3352-FDS/DGD
 PART B

1. What is Data Science; explain the benefits of data science.

Data
In computing, data is information that has been translated into a form that is efficient for
movement orprocessing
Data Science
Data science is an evolutionary extension of statistics capable of dealing with the massive
amounts of dataproduced today. It adds methods from computer science to the repertoire of
statistics.
Benefits and uses of data science
Data science and big data are used almost everywhere in both commercial and noncommercial
Settings
 Commercial companies in almost every industry use data science and big data to
gain insights into their customers, processes, staff, completion, and products.
 Many companies use data science to offer customers a better user experience, as
well as to cross-sell, up-sell, and personalize their offerings.
 Governmental organizations are also aware of data’s value. Many governmental
organizations not only rely on internal data scientists to discover valuable
information, but also share their data with the public.
 Nongovernmental organizations (NGOs) use it to raise money and defend their causes.
 Universities use data science in their research but also to enhance the study
experience of their students. The rise of massive open online courses (MOOC)
produces a lot of data, which allows universities to study how this type of learning
can complement traditional classes.
Advantages of Data Science
The various benefits of Data Science are as follows:
1. It’s in Demand
Data Science is greatly in demand. Prospective job seekers have numerous opportunities. It is
the fastest growing job on Linkedin and is predicted to create 11.5 million jobs by 2026. This
makes Data Science a highly employable job sector.
2. Abundance of Positions
There are very few people who have the required skill-set to become a complete Data Scientist.
This makes Data Science less saturated as compared with other IT sectors.
Therefore, Data Science is a vastly abundant field and has a lot of opportunities. The field of
Data Science is high in demand but low in supply of Data Scientists.
3. A Highly Paid Career
Data Science is one of the most highly paid jobs. According to Glassdoor, Data Scientists make
an average of $116,100 per year. This makes Data Science a highly lucrative career option.
4. Data Science is Versatile
There are numerous applications of Data Science. It is widely used in health-care, banking,
consultancy services, and e-commerce industries. Data Science is a very versatile field.
Therefore, you will have the opportunity to work in various fields.
5. Data Science Makes Data Better
Companies require skilled Data Scientists to process and analyze their data. They not only
analyze the data but also improve its quality. Therefore, Data Science deals with enriching data
and making it better for their company.

TEC/CSE/CS3352-FDS/DGD
Explore the Future of Data Science
6. Data Scientists are Highly Prestigious
Data Scientists allow companies to make smarter business decisions. Companies rely on Data
Scientists and use their expertise to provide better results to their clients. This gives Data
Scientists an important position in the company.
7. No More Boring Tasks
Data Science has helped various industries to automate redundant tasks. Companies are using
historical data to train machines in order to perform repetitive tasks. This has simplified the
arduous jobs undertaken by humans before.
8. Data Science Makes Products Smarter
Data Science involves the usage of Machine Learning which has enabled industries to create
better products tailored specifically for customer experiences.
For example, Recommendation Systems used by e-commerce websites provide personalized
insights to users based on their historical purchases. This has enabled computers to understand
human-behavior and take data-driven decisions.
9. Data Science can Save Lives
Healthcare sector has been greatly improved because of Data Science. With the advent of
machine learning, it has been made easier to detect early-stage tumors. Also, many other
health-care industries are using Data Science to help their clients.
10. Data Science Can Make You A Better Person
Data Science will not only give you a great career but will also help you in personal growth.
You will be able to have a problem-solving attitude. Since many Data Science roles bridge IT
and Management, you will be able to enjoy the best of both worlds.
Disadvantages of Data Science
While Data Science is a very lucrative career option, there are also various disadvantages to
this field. In order to understand the full picture of Data Science, we must also know the
limitations of Data Science. Some of them are as follows:
1. Data Science is Blurry Term
Data Science is a very general term and does not have a definite definition. While it has
become a buzzword, it is very hard to write down the exact meaning of a Data Scientist. A Data
Scientist’s specific role depends on the field that the company is specializing in.
While some people have described Data Science to be the fourth paradigm of Science, few
critics have called it a mere rebranding of Statistics.
2. Mastering Data Science is near to impossible
Being a mixture of many fields, Data Science stems from Statistics, Computer Science and
Mathematics. It is far from possible to master each field and be equivalently expert in all of
them.
While many online courses have been trying to fill the skill-gap that the data science industry is
facing, it is still not possible to be proficient at it considering the immensity of the field.
A person with a background in Statistics may not be able to master Computer Science on short
notice in order to become a proficient Data Scientist. Therefore, it is an ever-changing,
dynamic field that requires the person to keep learning the various avenues of Data Science.
3. Large Amount of Domain Knowledge Required
Another disadvantage of Data Science is its dependency on Domain Knowledge. A person with
a considerable background in Statistics and Computer Science will find it difficult to solve
Data Science problem without its background knowledge.
The same holds true for its vice-versa. For example, A health-care industry working on an
analysis of genomic sequences will require a suitable employee with some knowledge of

TEC/CSE/CS3352-FDS/DGD
genetics and molecular biology.
This allows the Data Scientists to make calculated decisions in order to assist the company.
However, it becomes difficult for a Data Scientist from a different background to acquire
specific domain knowledge. This also makes it difficult to migrate from one industry to
another.
4. Arbitrary Data May Yield Unexpected Results
A Data Scientist analyzes the data and makes careful predictions in order to facilitate the
decision-making process. Many times, the data provided is arbitrary and does not yield
expected results. This can also fail due to weak management and poor utilization of resources.
5. Problem of Data Privacy
For many industries, data is their fuel. Data Scientists help companies make data-driven
decisions. However, the data utilized in the process may breach the privacy of customers.
The personal data of clients are visible to the parent company and may at times cause data
leaks due to lapse in security. The ethical issues regarding preservation of data-privacy and its
usage have been a concern for many industries.

2. List the facets of data with example.

In data science and big data you’ll come across many different types of data, and each of them
tends to requiredifferent tools and techniques. The main categories of data are these:
 Structured
 Unstructured
 Natural language
 Machine-generated
 Graph-based
 Audio, video, and images
 Streaming
Structured data
 Structured data is data that depends on a data model and resides in a fixed field
within a record. As such, it’s often easy to store structured data in tables within
databases or Excel files
 SQL, or Structured Query Language, is the preferred way to manage and query data
that resides in databases.

TEC/CSE/CS3352-FDS/DGD
Unstructured data
Unstructured data is data that isn’t easy to fit into a data model because the content is context-
specific orvarying. One example of unstructured data is your regular email

Natural language
 Natural language is a special type of unstructured data; it’s challenging to process
because it requiresknowledge of specific data science techniques and linguistics.
 The natural language processing community has had success in entity recognition, topic
recognition, summarization, text completion, and sentiment analysis, but models trained
in one domain don’t generalize well to other domains.
 Even state-of-the-art techniques aren’t able to decipher the meaning of every piece of text.

TEC/CSE/CS3352-FDS/DGD
Machine-generated data
 Machine-generated data is information that’s automatically created by a
computer, process,application, or other machine without human intervention.
 Machine-generated data is becoming a major data resource and will continue to do so.
 The analysis of machine data relies on highly scalable tools, due to its high
volume and [Link] of machine data are web server logs, call detail
records, network event logs, and telemetry.

Graph-based or network data

 ―Graph data‖ can be a confusing term because any data can be shown in a graph.
 Graph or network data is, in short, data that focuses on the relationship or adjacency of
objects.
 The graph structures use nodes, edges, and properties to represent and store graphical
data.
 Graph-based data is a natural way to represent social networks, and its
structure allows you tocalculate specific metrics such as the influence of a person
and the shortest path between two people.

Audio, image, and video

 Audio, image, and video are data types that pose specific challenges to a data scientist.
 Tasks that are trivial for humans, such as recognizing objects in pictures, turn out to
be challenging forcomputers.
 MLBAM (Major League Baseball Advanced Media) announced in 2014 that
they’ll increase videocapture to approximately 7 TB per game for the purpose of
live, in-game analytics.
 Recently a company called DeepMind succeeded at creating an algorithm that’s
capable of learninghow to play video games.

TEC/CSE/CS3352-FDS/DGD
  This algorithm takes the video screen as input and learns to interpret
everything via a complexof deep learning.
Streaming data
 The data flows into the system when an event happens instead of being loaded
into a data store in abatch.
 Examples are the ―What’s trending‖ on Twitter, live sporting or music events, and the
stock market.

3. Briefly explain the steps in data science process with diagram.

The typical data science process consists of six steps through which you’ll iterate, as shown
in figure
1. The first step of this process is setting a research goal. The main purpose here is making
sure all the stakeholders understand the what, how, and why of the project. In every
serious project this will result in a project charter.
2. The second phase is data retrieval. You want to have data available for analysis, so this
step includes finding suitable data and getting access to the data from the data owner.
The result is data in its raw form, which probably needs polishing and transformation
before it becomes usable.
3. Now that you have the raw data, it’s time to prepare it. This includes transforming the
data from a raw form into data that’s directly usable in your models. To achieve this,

TEC/CSE/CS3352-FDS/DGD
 you’ll detect and correct different kinds of errors in the data, combine data from
different data sources, and transform it. If you have successfully completed this step,
you can progress to data visualization and modeling.
4. The fourth step is data exploration. The goal of this step is to gain a deep understanding
of the data. You’ll look for patterns, correlations, and deviations based on visual and
descriptive techniques. The insights you gain from this phase will enable you to start
modeling.
5. Finally, we get to model building (often referred to as ―data modeling‖ throughout this
book). It is now that you attempt to gain the insights or make the predictions stated in
your project charter. Now is the time to bring out the heavy guns, but remember
research has taught us that often (but not always) a combination of simple models tends
to outperform one complicated model. If you’ve done this phase right, you’re almost
done.
6. The last step of the data science model is presenting your results and automating the
analysis, if needed. One goal of a project is to change a process and/or make better
decisions. You may still need to convince the business that your findings will indeed
change the business process as expected. Thisis where you can shine in your influencer
role. The importance of this step is more apparent in projects on a strategic and tactical
level. Certain projects require you to perform the business process over and over again,
so automating the project will save time.

4. Explain the procedure about Data mining.

Data mining is the process of discovering actionable information from large sets of data. Data
mining uses mathematical analysis to derive patterns and trends that exist in data. Typically,
these patterns cannot be discovered by traditional data exploration because the relationships are
too complex or because there is too much data.
These patterns and trends can be collected and defined as a data mining model. Mining models
can be applied to specific scenarios, such as:
o Forecasting: Estimating sales, predicting server loads or server downtime
o Risk and probability: Choosing the best customers for targeted mailings,
determining the probablebreak-even point for risk scenarios, assigning probabilities to
diagnoses or other outcomes
o Recommendations: Determining which products are likely to be sold together,
generatingrecommendations
o Finding sequences: Analyzing customer selections in a shopping cart, predicting next likely
events
o Grouping: Separating customers or events into cluster of related items, analyzing
and predictingaffinities

Building a mining model is part of a larger process that includes everything from asking
questions about the data and creating a model to answer those questions, to deploying the
model into a working environment. Thisprocess can be defined by using the following six basic
steps:
1. Defining the Problem
2. Preparing Data
3. Exploring Data
4. Building Models

TEC/CSE/CS3352-FDS/DGD
 5. Exploring and Validating Models
6. Deploying and Updating Models

The following diagram describes the relationships between each step in the process, and the
technologies in Microsoft SQL Server that you can use to complete each step.

Defining the Problem

The first step in the data mining process is to clearly define the problem, and consider ways
that data can be utilized to provide an answer to the problem.

This step includes analyzing business requirements, defining the scope of the problem, defining
the metrics by which the model will be evaluated, and defining specific objectives for the data
mining project. These tasks translate into questions such as the following:
 What are you looking for? What types of relationships are you trying to find?
 Does the problem you are trying to solve reflect the policies or processes of the business?
 Do you want to make predictions from the data mining model, or just look for
interesting patterns and associations?
 Which outcome or attribute do you want to try to predict?
 What kind of data do you have and what kind of information is in each column? If there
are multiple tables, how are the tables related? Do you need to perform any cleansing,
aggregation, or processing tomake the data usable?
 How is the data distributed? Is the data seasonal? Does the data accurately represent the
processes of the business?

Preparing Data
 The second step in the data mining process is to consolidate and clean the data that
was identified in the Defining the Problem step.
 Data can be scattered across a company and stored in different formats, or may
contain inconsistenciessuch as incorrect or missing entries.
 Data cleaning is not just about removing bad data or interpolating missing values,
but about finding hidden correlations in the data, identifying sources of data that are
the most accurate, and determining which columns are the most appropriate for use
in analysis

TEC/CSE/CS3352-FDS/DGD
Exploring Data
Exploration techniques include calculating the minimum and maximum values, calculating
mean and standard deviations, and looking at the distribution of the data. For example, you
might determine by reviewing the maximum, minimum, and mean values that the data is not
representative of your customers or business processes, and that you therefore must obtain
more balanced data or review the assumptions that are the basis for your expectations. Standard
deviations and other distribution values can provide useful information about the stability and
accuracy of the results.

Building Models
The mining structure is linked to the source of data, but does not actually contain any data until
you process [Link] you process the mining structure, SQL Server Analysis Services generates
aggregates and other statistical information that can be used for analysis. This information can
be used by any mining model that is based on the structure.
Exploring and Validating Models
Before you deploy a model into a production environment, you will want to test how well the
model performs. Also, when you build a model, you typically create multiple models with
different configurations and test all models to see which yields the best results for your
problem and your data.
Deploying and Updating Models
After the mining models exist in a production environment, you can perform many tasks,
depending on your needs. The following are some of the tasks you can perform:
 Use the models to create predictions, which you can then use to make business decisions.
 Create content queries to retrieve statistics, rules, or formulas from the model.
 Embed data mining functionality directly into an application. You can include
Analysis Management Objects (AMO), which contains a set of objects that your
application can use to create, alter, process, and delete mining structures and mining
models.
 Use Integration Services to create a package in which a mining model is used to
intelligently separate incoming data into multiple tables.
 Create a report that lets users directly query against an existing mining model
 Update the models after review and analysis. Any update requires that you reprocess the
models.
 Update the models dynamically, as more data comes into the organization, and
making constant changes to improve the effectiveness of the solution should be part
of the deployment strategy.

5. Explain about data warehousing.

Data warehousing is the process of constructing and using a data warehouse. A data warehouse
is constructed by integrating data from multiple heterogeneous sources that support analytical
reporting, structured and/or ad hoc queries, and decision making. Data warehousing involves
data cleaning, data integration, and dataconsolidations.
Characteristics of data warehouse
The main characteristics of a data warehouse are as follows:
Subject-Oriented
A data warehouse is subject-oriented since it provides topic-wise information rather
than the overall processes of a business. Such subjects may be sales, promotion,
inventory, etc

TEC/CSE/CS3352-FDS/DGD
 Integrated
A data warehouse is developed by integrating data from varied sources into a
consistent format. The data must be stored in the warehouse in a consistent and
universally acceptable manner in terms of naming, format, and coding. This facilitates
effective data analysis.
Non-Volatile
Data once entered into a data warehouse must remain unchanged. All data is read-
only. Previous data is not erased when current data is entered. This helps you to
analyze what has happened and when.
Time-Variant
The data stored in a data warehouse is documented with an element of time, either
explicitly or implicitly. An example of time variance in Data Warehouse is exhibited in
the Primary Key, which must have an element of time like the day, week, or month.

Database vs. Data Warehouse

Although a data warehouse and a traditional database share some similarities, they need not be
the same idea. The main difference is that in a database, data is collected for multiple
transactional purposes. However, in a data warehouse, data is collected on an extensive scale to
perform analytics. Databases provide real-time data, while warehouses store data to be accessed
for big analytical queries.
Data Warehouse Architecture
Usually, data warehouse architecture comprises a three-tier structure.
Bottom Tier
The bottom tier or data warehouse server usually represents a relational database system.
Back-end tools areused to cleanse, transform and feed data into this layer.
Middle Tier
The middle tier represents an OLAP server that can be implemented in two ways.
The ROLAP or Relational OLAP model is an extended relational database management system
that mapsmultidimensional data process to standard relational process.
The MOLAP or multidimensional OLAP directly acts on multidimensional data and operations.
Top Tier
This is the front-end client interface that gets data out from the data warehouse. It holds various
tools likequery tools, analysis tools, reporting tools, and data mining tools.
How Data Warehouse Works
Data Warehousing integrates data and information collected from various sources into one
comprehensive database. For example, a data warehouse might combine customer information
from an organization’s point- of-sale systems, its mailing lists, website, and comment cards.
It might also incorporate confidential information about employees, salary information, etc.
Businesses use such components of data warehouse to analyze customers.

Data mining is one of the features of a data warehouse that involves looking for meaningful
data patterns in vast volumes of data and devising innovative strategies for increased sales and
profits.
Types of Data Warehouse
There are three main types of data warehouse.

TEC/CSE/CS3352-FDS/DGD
 Enterprise Data Warehouse (EDW)
This type of warehouse serves as a key or central database that facilitates decision-
support services throughout the enterprise. The advantage to this type of warehouse is
that it provides access to cross-organizational information, offers a unified approach to
data representation, and allows running complex queries.

Operational Data Store (ODS)

This type of data warehouse refreshes in real-time. It is often preferred for routine
activities like storing employee records. It is required when data warehouse systems do
not support reporting needs of the business.

Data Mart
A data mart is a subset of a data warehouse built to maintain a particular department,
region, or business unit. Every department of a business has a central repository or data
mart to store data. The data from the data mart is stored in the ODS periodically. The
ODS then sends the data to the EDW, where it is stored and used.

6. Explain about Basic Statistical descriptions of Data

1. Descriptive Statistics
It is used to describe the basic features of data that provide a summary of the given data set
which can either represent the entire population or a sample of the population. It is derived
from calculations that include:
 Mean: It is the central value which is commonly known as arithmetic average.
 Mode: It refers to the value that appears most often in a data set.
 Median: It is the middle value of the ordered set that divides it in exactly half.
2. Variability
Variability includes the following parameters:
 Standard Deviation: It is a statistic that calculates the dispersion of a data set as
compared to its mean.
 Variance: It refers to a statistical measure of the spread between the numbers in a data
set. In general terms, it means the difference from the mean. A large variance indicates
that numbers are far apart from the mean or average value. Small variance indicates that
the numbers are closer to the average values. Zero variance indicates that the values are
identical to the given set.
 Range: This is defined as the difference between the largest and smallest value of a
dataset.
 Percentile: It refers to the measure used in statistics that indicates the value below
which the given percentage of observation in the dataset falls.
 Quartile: It is defined as the value that divides the data points into quarters.
 Interquartile Range: It measures the middle half of your data. In general terms, it is the
middle 50% of the dataset.
3. Correlation
It is one of the major statistical techniques that measure the relationship between two
variables. The correlation coefficient indicates the strength of the linear relationship
between two variables.
 A correlation coefficient that is more than zero indicates a positive relationship.
 A correlation coefficient that is less than zero indicates a negative relationship.

TEC/CSE/CS3352-FDS/DGD
 Correlation coefficient zero indicates that there is no relationship between the two
variables.
4. Probability Distribution
It specifies the likelihood of all possible events. In simple terms, an event refers to the
result of an experiment like tossing a coin. Events are of two types dependent and
independent.
 Independent event: The event is said to be an Independent event when it is not affected
by the earlier events. For example, tossing a coin, let us consider a coin is tossed the
first outcome is head when the coin is tossed again the outcome may be head or tail. But
this is entirely independent of the first trial.
 Dependent event: The event is said to be dependent when the occurrence of the event is
dependent on the earlier events. For example when a ball is drawn from a bag that
contains red and blue balls. If the first ball drawn is red, then the second ball may be red
or blue; this depends on the first trial.
The probability of independent events is calculated by simply multiplying the probability of
each event and for a dependent event is calculated by conditional probability.
5. Regression
It is a method that is used to determine the relationship between one or more independent
variables and a dependent variable. Regression is mainly of two types:
 Linear regression: It is used to fit the regression model that explains the relationship
between a numeric predictor variable and one or more predictor variables.
 Logistic regression: It is used to fit a regression model that explains the relationship
between the binary response variable and one or more predictor variables.
6. Normal Distribution
Normal is used to define the probability density function for a continuous random variable
in a system. The standard normal distribution has two parameters – mean and standard
deviation that are discussed above. When the distribution of random variables is unknown,
the normal distribution is used. The central limit theorem justifies why normal distribution
is used in such cases.
7. Bias
In statistical terms, it means when a model is representative of a complete population. This
needs to be minimized to get the desired outcome.
The three most common types of bias are:
 Selection bias: It is a phenomenon of selecting a group of data for statistical analysis,
the selection in such a way that data is not randomized resulting in the data being
unrepresentative of the whole population.
 Confirmation bias: It occurs when the person performing the statistical analysis has
some predefined assumption.
 Time interval bias: It is caused intentionally by specifying a certain time range to favor
a particular outcome.

TEC/CSE/CS3352-FDS/DGD
 CS3352 - FOUNDATION
UNIT –OFI DATA SCIENCE
Question Bank
INTRODUCTION
Unit – II DESCRIBING DATA

Types of Data - Types of Variables -Describing Data with Tables and Graphs –Describing Data
withAverages - Describing Variability - Normal Distributions and Standard (z) Scores

1. What is Frequency distribution?

Frequency distribution is a collection of observations produced by sorting observations into classesand
showing their frequency (f) of occurrence in each class.
2. What are the types and uses of Frequency distributions?
Types of Frequency distributions:
 Grouped frequency distribution.
 Ungrouped frequency distribution.
 Cumulative frequency distribution.
 Relative frequency distribution.
 Relative cumulative frequency distribution.
Uses:
A frequency distribution helps us to detect any pattern in the data (assuming a pattern exists) by
superimposing some order on the inevitable variability among observations.
3. What is Grouped frequency distribution:
When observations are sorted into classes of more than one value, the result is referred to as a
frequency distribution for grouped data.
4. What is ungrouped frequency distribution?
When observations are sorted into classes of single values the result is referred to as a frequency
distribution for ungrouped data.
5. What is cumulative frequency distribution?
Cumulative frequency distributions show the total number of observations in each class and in all
lower-ranked classes. This type of distribution can be used effectively with sets of scores, such as test
scores for intellectual or academic aptitude, when relative standing within the distribution assumes
primary importance.
6. What is relative frequency distribution?
Relative frequency distributions show the frequency of each class as a part or fraction of the total
frequency for the entire distribution.
7. Define Percentile Ranks?
The percentile rank of a score indicates the percentage of scores in the entire distribution with similar
or smaller values than that score.
8. What is Histograms?
A histogram is the most commonly used graph to show frequency distributions. It is a bar-type graph
for quantitative data. The common boundaries between adjacent bars emphasize the continuity of the
data, as with continuous variables.
9. Explain any three Features of histograms?
Equal units along the horizontal axis (the X axis, or abscissa) reflect the various class intervals of the
frequency distribution. Equal units along the vertical axis (the Y axis, or ordinate) reflect increases in
frequency. (The units along the vertical axis do not have to be the same width as those along the

TEC/CSE/CS3352-FDS/DGD
horizontal axis.). The body of the histogram consists of a series of bars whose heights reflect the
frequencies for the various classes.
10. What is Frequency Polygon?
Frequency Polygon is a line graph for quantitative data that also emphasizes the continuity of
continuous variables. Frequency polygons may be constructed directly from frequency distributions.
11. What is mean?
The mean is found by adding all scores and then dividing by the number of scores.
MEAN = SUM OF ALL SCORES /NUMBER OF SCORES
12. What is median?
The median reflects the middle value when observations are ordered from least to most. The median
splits a set of ordered observations into two equal parts, the upper and lower halves. In other words,
the median has a percentile rank of 50, since observations with equal or smaller values constitute 50
percent of the entire distribution.
13. What is mode?
Mode reflects the value of the most frequently occurring [Link] is easy to assign a value to the mode
if the data are organized.
14. What if a distribution have More than one mode or no mode at all?
Distributions can have more than one mode (or no mode at all). Distributions with two obvious peaks,
even though they are not exactly the same height, are referred to as bimodal. Distributions with more
than two peaks are referred to as multimodal. The presence of more than one mode might reflect
important differences among subsets of data.
15. Explain Range, variance and standard deviation?
Range: The range is the difference between the largest and smallest scores
Variance: The variance is a measure of variability
Standard deviation: Standard deviation is a measure of the amount of variations or dispersion of a
set of value. A low standard deviation indicates that the values tend to be close to the mean of the set,
while high standard deviation indicates that the values are spread out over a wider range.
𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛 = √variance
16. What is DEGREES OF FREEDOM (df)?
Degrees of freedom (df) refers to the number of values that are free to vary, given one or more
mathematical restrictions, in a sample being used to estimate a population characteristic.
17. What is INTERQUARTILE RANGE (IQR)?
Interquartile range (IQR), is simply the range for the middle 50 percent of the scores. More
specifically, the IQR equals the distance between the third quartile (or 75th percentile) and the first
quartile.
18. What is Qualitative Data?
Qualitative data consist of words (Yes or No), letters (Y or N), or numerical codes (0 or 1) that
represent a class or category.
19. What is Ranked Data?
Ranked data consist of numbers (1st, 2nd, . . . 40th place) that represent relative standing within a
group.
20. What is Quantitative Data?
Quantitative data consist of numbers (weights of 238, 170, . . . 185 lbs) that represent an amount or a
count. To determine the type of data, focus on a single observation in any collection of observations

TEC/CSE/CS3352-FDS/DGD
 PART B
1. Explain about different variables with example.
A variable is a characteristic or property that can take on different values.
The weights can be described not only as quantitative data but also as observations for a
quantitativevariable, since the various weights take on different numerical values.
By the same token, the replies can be described as observations for a qualitative variable, since
thereplies to the Facebook profile question take on different values of either Yes or No.
Given this perspective, any single observation can be described as a constant, since it takes on
onlyone value.
Discrete and Continuous Variables
Quantitative variables can be further distinguished as discrete or continuous.
A discrete variable consists of isolated numbers separated by gaps.
Discrete variables can only assume specific values that you cannot subdivide. Typically, you
count discretevalues, and the results are integers.
Examples
 Counts- such as the number of children in a family. (1, 2, 3, etc., but never 1.5)
 These variables cannot have fractional or decimal values. You can have 20 or 21 cats, but not
20.5
 The number of heads in a sequence of coin tosses.
 The result of rolling a die.
 The number of patients in a hospital.
The population of a country.
While discrete variables have no decimal places, the average of these values can be fractional. For
example, families can have only a discrete number of children: 1, 2, 3, etc. However, the average
number of children per family can be 2.2.
A continuous variable consists of numbers whose values, at least in theory, have no restrictions.
Continuous variables can assume any numeric value and can be meaningfully split into smaller
parts. Consequently, they have valid fractional and decimal values. In fact, continuous variables
have an infinite number of potential values between any two points. Generally, you measure them
using a scale.
Examples of continuous variables include weight, height, length, time, and temperature.
Durations, such as the reaction times of grade school children to a fire alarm; and standardized test
scores, such as those on the Scholastic Aptitude Test (SAT).
Independent and Dependent
VariablesIndependent
Variable
In an experiment, an independent variable is the treatment manipulated by the investigator.
 Independent variables (IVs) are the ones that you include in the model to explain or predict
changes inthe dependent variable.
 Independent indicates that they stand alone and other variables in the model do not influence
them.
 Independent variables are also known as predictors, factors, treatment variables,
explanatory variables, input variables, x-variables, and right-hand variables—because they
appear on the right side of the equals sign in a regression equation.
 It is a variable that stands alone and isn't changed by the other variables you are trying to

TEC/CSE/CS3352-FDS/DGD
 measure.
 For example, someone's age might be an independent variable. Other factors (such as what
they eat, how much they go to school, how much television they watch)
 The impartial creation of distinct groups, which differ only in terms of the independent
variable, has a most desirable consequence. Once the data have been collected, any
difference between the groups can be interpreted as being caused by the independent
variable.
Dependent Variable
When a variable is believed to have been influenced by the independent variable, it is called a
dependent variable. In an experimental setting, the dependent variable is measured, counted, or
recorded by the investigator.
 The dependent variable (DV) is what you want to use the model to explain or predict.
The values ofthis variable depend on other variables.
 It’s also known as the response variable, outcome variable, and left-hand variable.
Graphs placedependent variables on the vertical, or Y, axis. a dependent variable is exactly
what it sounds like. It is something that depends on other factors.
 For example the blood sugar test depends on what food you ate, at which time you ate etc.
 Unlike the independent variable, the dependent variable isn’t manipulated by the
investigator. Instead, it represents an outcome: the data produced by the experiment.
Confounding Variable
An uncontrolled variable that compromises the interpretation of a study is known as a
confounding variable. Sometimes a confounding variable occurs because it’s impossible to assign
subjects randomly to different conditions.

2. Explain the different types of frequency distribution with example.

RELATIVE FREQUENCY DISTRIBUTIONS

Relative frequency distributions show the frequency
of each class as a part or fraction of the total
frequency for the entiredistribution.
This type of distribution is especially helpful when
you mustcompare two or more distributions based on
different total numbers of observations.
The conversion to relative frequencies allows a
direct comparison of the shapes of two
distributions withoutadjust other observations.
Constructing Relative Frequency Distributions
To convert a frequency distribution into a relative
frequency distribution, divide the frequency for each
class by the total f requency for the entire
distribution.
Table 2.5 illustrates a relative frequency distribution
based onthe weight distribution of Table 2.2.
Percentages or Proportions
Some people prefer to deal with percentages rather than proportions because percentages usually

TEC/CSE/CS3352-FDS/DGD
lack decimal points. A proportion always varies between 0 and 1, whereas a percentage always
varies between 0 percent and 100 percent.
To convert the relative frequencies, multiply each proportion by 100; that is, move the decimal
point two places to the right.

CUMULATIVE FREQUENCY DISTRIBUTIONS

Cumulative frequency distributions show the total number of observations in each class and in all
lower- ranked classes.
Cumulative frequencies are usually converted, in turn, to cumulative percentages. Cumulative
percentagesare often referred to as percentile ranks.
Constructing Cumulative Frequency Distributions
To convert a frequency distribution into a cumulative frequency distribution, add to the frequency
of each class the sum of the frequencies of all classes ranked below it.

5
Cumulative Percentages
As has been suggested, if relative standing within a distribution is particularly important, then
cumulativefrequencies are converted to cumulative percentages
To obtain this cumulative percentage, the cumulative frequency of the class should be divided by the
totalfrequency of the entire distribution.
Percentile Ranks
When used to describe the relative position of any score within its parent distribution, cumulative
percentages are referred to as percentile ranks.
The percentile rank of a score indicates the percentage of scores in the entire distribution with
similar or smaller values than that score. Thus a weight has a percentile rank of 80 if equal or
lighter weights constitute 80 percent of the entire distribution.
FREQUENCY DISTRIBUTIONS FOR QUALITATIVE (NOMINAL) DATA
Frequency distributions for qualitative data are easy to
construct. Simply determine the frequency with which
observations occupy Each class, and report these
frequencies as shown in Table 2.7 forthe Face book
profile survey

TEC/CSE/CS3352-FDS/DGD
Qualitative data have an ordinal level of measurement
because Observations can be ordered from least to most,
that order shouldbe preserved in the frequency table
Relative and Cumulative Distributions for Qualitative Data
Frequency distributions for qualitative variables can always be converted into relative frequency
distributions.
if measurement is ordinal because observations can be ordered from least to most, cumulative
frequencies(and cumulative percentages) can be used.

3. Describe Mode, Median, Mean and Averages with example.

MODE
The mode reflects the value of the most frequently occurring score. In other words
A mode is defined as the value that has a higher frequency in a given set of values. It is the value
that appearsthe most number of times.
Example:
In the given set of data: 2, 4, 5, 5, 6, 7, the mode of the data set is 5 since it has appeared in the set twice.
Types of Modes
Bimodal, Trimodal & Multimodal (More than one mode)
When there are two modes in a data set, then the set is called bimodal
For example, The mode of Set A = {2,2,2,3,4,4,5,5,5} is 2 and 5, because both 2 and 5 is repeated
three timesin the given set.
When there are three modes in a data set, then the set is called trimodal
For example, the mode of set A = {2,2,2,3,4,4,5,5,5,7,8,8,8} is 2, 5 and 8
When there are four or more modes in a data set, then the set is called multimodal
Example: The following table represents the number of wickets taken by a bowler in 10
matches. Find themode of the given set of data.

It can be seen that 2 wickets were taken by the bowler frequently in different matches. Hence, the
mode of thegiven data is 2.

TEC/CSE/CS3352-FDS/DGD
MEDIAN
The median reflects the middle value when observations are ordered from least to most.
The median splits a set of ordered observations into two equal parts, the upper and lower halves.
Finding the Median
Order scores from least to most.
If the total number of observation given is odd, then the formula to calculate the
median is:Median = {(n+1)/2}th term / observation
If the total number of observation is even, then the median
formula is:Median = 1/2[(n/2)th term + {(n/2)+1}th term ]
Example 1:
Find the median of the following:
4, 17, 77, 25, 22, 23, 92, 82, 40, 24, 14, 12, 67, 23, 29
Solution:
n= 15
When we put those numbers in the order we have:
4, 12, 14, 17, 22, 23, 23, 24, 25, 29, 40, 67, 77, 82, 92,
Median = {(n+1)/2}th term
= (15+1)/2
=8
The 8th term in the list is 24
The median value of this set of numbers is 24.
Example 2:
Find the median of the following:
9,7,2,11,18,12,6,4
S
o
l
u
t
i
o
n

n
=
8
When we put those numbers in the order we have:
2, 4, 6, 7, 9,11, 12, 18
Median = 1/2[(n/2)th term + {(n/2)+1}th term ]
= ½ [(8/2) term + ((8/2)+1)term]
=1/2[4th term+5th term] (in our list 4th term is 7 and 5th term is 9)
= ½[7+9]
=1/2(16)
=8
The median value of this set of numbers is 8.

TEC/CSE/CS3352-FDS/DGD
MEAN
The mean is found by adding all scores and then dividing by the number of scores.
Mean is the average of the given numbers and is calculated by dividing the sum of given numbers by
the totalnumber of numbers.

Types of means
 Sample mean
 Population mean

Sample Mean
The sample mean is a central tendency measure. The arithmetic average is computed using
samples or random values taken from the population. It is evaluated as the sum of all the sample
variables divided by the total number of variables.

Population Mean
The population mean can be calculated by the sum of all values in the given data/population
divided by a totalnumber of values in the given data/population.

AVERAGES FOR QUALITATIVE AND RANKED DATA

Mode
The mode always can be used with qualitative data.
Median
The median can be used whenever it is possible to order qualitative data from least to most
because the levelof measurement is ordinal.

4. Specify about the Normal Distributions and Standard (z) Scores.

THE NORMAL CURVE
The normal distribution is a continuous probability distribution that is symmetrical on both sides of
the mean, so the right side of the center is a mirror image of the left side.
Properties of the Normal Curve
 The normal curve is a theoretical curve defined for a continuous variable, as
described in Section 1.6, and noted for its symmetrical bell-shaped form, as revealed in below
figure
 Because the normal curve is symmetrical, its lower half is the mirror image of its upper
half.
 The normal curve peaks above a point midway along the horizontal spread
and then tapers off gradually in either direction from the peak (without actually touching the

TEC/CSE/CS3352-FDS/DGD
horizontal axis, since, in theory, the tails of a normal curve extend infinitely far).
 The values of the mean, median (or 50th percentile), and mode, located at a point
midway along the horizontal spread, are the same for the normal curve.
Properties of a normal distribution
 The mean, mode and median are all equal.
 The curve is symmetric at the center (i.e. around the mean, μ).
 Exactly half of the values are to the left of center and exactly half the values are to the
right.
 The total area under the curve is 1.

Different Normal Curves

As a theoretical exercise, it is instructive to note the various types of normal curves that are
produced by an arbitrary change in the value of either the mean (μ) or the standard deviation (σ).
Obvious differences in appearance among normal curves are less important than you might
suspect. Because of their common mathematical origin, every normal curve can be interpreted in
exactly the same wayonce any distance from the mean is expressed in standard deviation units.

z SCORES

A z score is a unit-free, standardized score that, regardless of the original units of measurement,
indicates howmany standard deviations a score is above or below the mean of its distribution.
A z score can be defined as a measure of the number of standard deviations by which a score
is below orabove the mean of a distribution. In other words, it is used to determine the distance of
a score from the mean. If the z score is positive it indicates that the score is above the mean. If it is

TEC/CSE/CS3352-FDS/DGD
negative then the score will be below the mean. However, if the z score is 0 it denotes that the data
point is the same as the mean.
To obtain a z score, express any original score, whether measured in inches, milliseconds, dollars,
IQ points, etc., as a deviation from its mean (by subtracting its mean) and then split this deviation
into standard deviationunits (by dividing by its standard deviation),

Where X is the original score and μ and σ are the mean and the standard deviation,
respectively, for the normal distribution of the original scores. Since identical units of
measurement appear in both the numerator and denominator of the ratio for z, the original
units of measurement cancel each other and the z score emerges as a unit-free or standardized
number, often referred to as a standard score.
A z score consists of two parts:
1. A positive or negative sign indicating whether it’s above or below the mean; and
2. A number indicating the size of its deviation from the mean in standard deviation units.
Converting
to z Scores
Example
Suppose on a GRE test a score of 1100 is obtained. The mean score for the GRE test is 1026 and
the population standard deviation is 209. In order to find how well a person scored with respect to
the score of an average test taker, the z score will have to be determined.

The steps to calculate the z score are as follows:

 Step 1: Write the value of the raw score in the z score equation. z = (1100−μ) /σ
 Step 2: Write the mean and standard deviation of the population in
the z score formula.z = (1100−1026) / 209
 Step 3: Perform the calculations to get the required z score. z = 0.345
 Step 4: A z score table can be used to find the percentage of test-takers that are
below the score of theperson. Using the first two digits of the z score, determine the row
containing these digits of the z table. Now using the 2nd digit after the decimal, find the
corresponding column. The intersection of this row and column will give a value. As shown
below, this value will be 0.6368 for the given example.
 Step 5: Use the value from step 5 and multiply it by 100 to get the required
percentage. 0.6368 * 100 =63.68%. This shows that 63.68% of test-takers scores are lesser than
the given raw score.
STANDARD NORMAL CURVE
If the original distribution approximates a normal curve, then the shift to standard or z scores will
always produce a new distribution that approximates the standard normal curve. This is the one
normal curve for which a table is actually available.

Although there is an infinite number of different normal curves, each with its own mean
and standarddeviation, there is only one standard normal curve, with a mean of 0 and a standard
deviation of 1.
For a standard normal curve

TEC/CSE/CS3352-FDS/DGD
Mean = 0

Standard deviation = 1

Standard Normal Table

The standard normal table consists of columns of z scores coordinated with columns of proportions
Using the Top Legend of the Table
Notice that columns are arranged in sets of three, designated as A, B, and C in the legend at the top
of the table. When using the top legend, all entries refer to the upper half of the standard normal
curve. The entries in column A are z scores, beginning with 0.00 and ending with 4.00

5. Explain about Finding Proportions and Finding Scores in detail.

Finding Proportions for One Score
 Sketch a normal curve and shade in the target area,
 Plan your solution according to the normal table.
 Convert X to z.

 Find the target area.

Finding Proportions between Two Scores

 Sketch a normal curve and shade in the target area, (example, find
proportion between 245to 255)
 Plan your solution according to the normal table.
 Convert X to z by expressing 255 as

TEC/CSE/CS3352-FDS/DGD
 Find the target area.

FINDING SCORES
So far, we have concentrated on normal curve problems for which Table A must be consulted to
findthe unknown proportion (of area) associated with some known score or pair of known scores
Now we will concentrate on the opposite type of normal curve problem for which Table A
must beconsulted to find the unknown score or scores associated with some known proportion.
For this type of problem requires that we reverse our use of Table A by entering
proportions incolumns B, C, B′, or C′ and finding z scores listed in columns A or A′.
Finding One Score
 Sketch a normal curve and, on the correct side of the mean, draw a line
representing the targetscore, as in figure

It’s often helpful to visualize the target score as splitting the total area into two sectors—one to the
left of (below) the target score and one to the right of (above) the target score
 Plan your solution according to the normal table.

TEC/CSE/CS3352-FDS/DGD
In problems of this type, you must plan how to find the z score for the target score. Because the
target score is on the right side of the mean, concentrate on the area in the upper half of the normal
curve, as described in columns B and C.
 Find z.
 Convert z to the target score.

When converting z scores to original scores, you will probably find it more efficient to use the
following equation

Finding Two Scores

 Sketch a normal curve. On either side of the mean, draw two lines representing the
two target scores,as in figure

 Plan your solution according to the normal table.

 Find z.
 Convert z to the target score.

Given a z score of zero or more, columns B and C indicate how the z score splits the area in the
upper half of the normal curve. As suggested by the shading in the top legend, column B
indicates the proportion of areabetween the mean and the z score, and column C indicates the
proportion of area beyond the z score, in theupper tail of the standard normal curve.
Using the Bottom Legend of the Table
Now the columns are designated as A′, B′, and C′ in the legend at the bottom of the table. When
using the bottom legend, all entries refer to the lower half of the standard normal curve.

TEC/CSE/CS3352-FDS/DGD
A negative z score, columns B′ and C′ indicate how that z score splits the lower half of the normal
curve. As suggested by the shading in the bottom legend of the table, column B′ indicates the
proportion of area between the mean and the negative z score, and column C′ indicates the
proportion of area beyond the negative z score,in the lower tail of the standard normal curve.

6. Analyze how graph can be used to represent qualitative and quantitative data?
Data can be described clearly and concisely with the aid of a well-constructed frequency
distribution. Anddata can often be described even more vividly by converting frequency
distributions into graphs.
GRAPHS FOR QUANTITATIVE DATA
Histograms
A bar-type graph for quantitative data. The common boundaries between adjacent bars
emphasize thecontinuity of the data, as with continuous variables.

TEC/CSE/CS3352-FDS/DGD
A histogram is a display of statistical information that uses rectangles to show the frequency of
data itemsin successive numerical intervals of equal size.

Important features of histograms

 Equal units along the horizontal axis (the X axis, or abscissa) reflect the various
class intervals of thefrequency distribution.
 Equal units along the vertical axis (the Y axis, or ordinate) reflect increases in
frequency. (The unitsalong the vertical axis do not have to be the same width as those along the
horizontal axis.)
 The intersection of the two axes defines the origin at which both numerical scales equal 0.
 Numerical scales always increase from left to right along the horizontal axis and
from bottom to topalong the vertical axis
 The body of the histogram consists of a series of bars whose heights reflect the
frequencies for thevarious classes.
 The adjacent bars in histograms have common boundaries that emphasize the
continuity ofquantitative data for continuous variables.
 The introduction of gaps between adjacent bars would suggest an artificial
disruption in the data moreappropriate for discrete quantitative variables or for qualitative
variables.

Figure: Histogram

Frequency Polygon
An important variation on a histogram is the frequency polygon, or line graph. Frequency
polygons maybe constructed directly from frequency distributions.

TEC/CSE/CS3352-FDS/DGD
Step-by-step transformation of a histogram into a frequency polygon

A. This panel shows the histogram for the weight distribution.

B. Place dots at the midpoints of each bar top or, in the absence of bar tops, at midpoints for
classes onthe horizontal axis, and connect them with straight lines.
C. c. Anchor the frequency polygon to the horizontal axis. First, extend the upper tail to the
midpoint of the first unoccupied class on the upper flank of the histogram. Then extend the
lower tail to the midpoint of the first unoccupied class on the lower flank of the histogram.
Now all of the area under the frequency polygon is enclosed completely.
D. Finally, erase all of the histogram bars, leaving only the frequency polygon.

Stem and Leaf Displays

Another technique for summarizing quantitative data is a stem and leaf display. Stem and leaf
displays are ideal for summarizing distributions, such as that for weight data, without destroying

TEC/CSE/CS3352-FDS/DGD
the identities of individual observations.
Constructing Stem and Leaf Display
The leftmost panel of table re-creates the weights.
To construct the stem and leaf display for the table given below, first note that, when counting by
tens, the weights range from the 130s to the 240s.
Arrange a column of numbers, the stems, beginning with 13 (representing the 130s) and ending
with 24 (representing the 240s). Draw a vertical line to separate the stems, which represent
multiples of 10, from the space to be occupied by the leaves, which represent multiples of 1.
For example
Enter each raw score into the stem and leaf display. As suggested by the shaded coding in Table
2.9, the first raw score of 160 reappears as a leaf of 0 on a stem of 16. The next raw score of 193
reappears as a leaf of 3 on a stem of 19, and the third raw score of 226 reappears as a leaf of 6 on a
stem of 22, and so on, until each raw score reappears as a leaf on its appropriate stem.

TYPICAL SHAPES
Whether expressed as a histogram, a frequency polygon, or a stem and leaf display, an important
characteristic of a frequency distribution is its shape. Below figure shows some of the more typical
shapes for smoothed frequency polygons (which ignore the inevitable irregularities of real data).

TEC/CSE/CS3352-FDS/DGD
A GRAPH FOR QUALITATIVE (NOMINAL) DATA
 As with histograms, equal segments along the horizontal axis are allocated to the
different words or classes that appear in the frequency distribution for qualitative data. Likewise,
equal segments along the vertical axis reflect increases in frequency. The body of the bar graph
consists of a series of bars whose heights reflect the frequencies for the various words or classes.
 A person’s answer to the question ―Do you have a Facebook profile?‖ is either Yes
or No, not some impossible intermediate value, such as 40 percent Yes and 60 percent No.
 Gaps are placed between adjacent bars of bar graphs to emphasize the
discontinuous nature ofqualitative data.

MISLEADING GRAPHS
Graphs can be constructed in an unscrupulous manner to support a particular point of view.
Popular sayings says, including ―Numbers don’t lie, but statisticians do‖ and ―There are three
kinds of lies—lies, damned lies, and statistics.‖

TEC/CSE/CS3352-FDS/DGD
 7. Explain how to Describe Variability in data science.
The range is the difference between the largest and smallest scores.
The range in statistics for a given data set is the difference between the highest and
lowest values. Forexample, if the given data set is {2,5,8,10,3}, then the range will be 10 – 2 =
8.
Example 1: Find the range of given observations: 32, 41, 28, 54, 35, 26,
23, 33, 38, [Link]: Let us first arrange the given values in ascending
order.
23, 26, 28, 32, 33, 35, 38, 40, 41, 54
Since 23 is the lowest value and 54 is the highest value, therefore, the range of the
observations will be;Range (X) = Max (X) – Min (X)
= 54 – 23
= 31
VARIANCE
Variance is a measure of how data points differ from the mean. A variance is a measure of
how far a set ofdata (numbers) are spread out from their mean (average) value.
Formula
σ = Σ(x-μ)2 or
Variance = (Standard deviation)2= σ2 = > σ 2= Σ(x-μ)2 /n
the values of all scores must be added and then divided by the total
number of [Link]
X = 5, 8, 6, 10, 12, 9, 11, 10, 12, 7
Solution
Me
an
=
su
m
(x)/
n
n=
10
sum (x) = 5+8+6+10+12+9+11+10+12+ 7
= 90
Mean=>
μ = 90 /
10 = 9
Deviation
from
mean
x- μ = -4, -1, -3, 1, 3, 0, 2,1,3,-2
(x-μ)2 = 16,1,9,1,9,0,4,1,9,4
Σ(x-μ)2 = 16+1+9+1+9+0+4+1+9+4
=54
σ 2= Σ(x-μ)2 /n
=54/10
= 5.4

TEC/CSE/CS3352-FDS/DGD
STANDARD DEVIATION
The standard deviation, the square root of the mean of all squared deviations from the
mean, that is,
Standard deviation = √variance
Standard Deviation: A rough measure of the average (or standard) amount by which
scores deviate

Standard Deviation: A Measure of Distance

The mean is a measure of position, but the standard deviation is a measure of distance (on
either side of themean of the distribution).
Sum of Squares (SS)
Calculating the standard deviation requires that we obtain first a value for the variance.
However, calculatingthe variance requires, in turn, that we obtain the sum of the squared
deviation scores.
The sum of squared deviation scores or more simply the sum of squares, symbolized by SS

―The sum of squares equals the sum of all squared deviation scores.‖ You can reconstruct this
formula byremembering the following three steps:
1. Subtract the population mean, μ, from each original score, X, to obtain a deviation score, X
− μ.
2. Square each deviation score, (X − μ)2, to eliminate negative signs.
3. Sum all squared deviation scores, Σ (X − μ)2.

TEC/CSE/CS3352-FDS/DGD
Sum of Squares Formulas for Sample
Sample notation can be substituted for population notation in the above two formulas
without causing anyessential changes:

TEC/CSE/CS3352-FDS/DGD
DEGREES OF FREEDOM (df)
 Degrees of freedom (df) refers to the number of values that are free to vary,
given one or more mathematical restrictions, in a sample being used to estimate a population
characteristic.

TEC/CSE/CS3352-FDS/DGD
 Degrees of freedom are the number of independent variables that can be
estimated in a statistical analysis. These values of these variables are without constraint,
although the values do impost restrictions on other variables if the data set is to comply with
estimate parameters.
 Degrees of Freedom (df ) The number of values free to vary, given one or more
mathematical restrictions.

Formula
Degree of freedom df = n-1

Example
Consider a data set consists of five positive integers. The sum of the five integers must be
the multiple of [Link] values are randomly selected as 3, 8, 5, and 4.
The sum of this for values is 20. So we have to choose the fifth integer to make the sum
divisible by [Link] the fifth element is 10.

The number of degrees of Degrees of Freedom (df ) The number of values free to vary, given
one or more mathematical restrictions. Freedom—in the numerator, as in the formulas for s2

and s. In fact, we can use degrees of freedom to rewrite the formulas for the sample variance
and standard deviation:

INTERQUARTILE RANGE (IQR)

The interquartile range (IQR), is simply the range for the middle 50 percent of the scores. More
specifically, the IQR equals the distance between the third quartile (or 75th percentile) and the
first quartile (or 25th percentile), that is, after the highest quarter (or top 25 percent) and the
lowest quarter (or bottom 25 percent) have been trimmed from the original set of scores. Since
most distributions are spread more widely in their extremities than their middle, the IQR tends
to be less than half the size of the range.

Simply, The IQR describes the middle 50% of values when ordered from lowest to highest. To
find the interquartile range (IQR), first find the median (middle value) of the lower and upper
half of the data. These values are quartile 1 (Q1) and quartile 3 (Q3). The IQR is the difference
between Q3 and Q1.

TEC/CSE/CS3352-FDS/DGD
TEC/CSE/CS3352-FDS/DGD
 UNITOF
CS3352 - FOUNDATION – I DATA SCIENCE
INTRODUCTION
Question Bank

Unit III - DESCRIBING RELATIONSHIPS

Correlation – Scatter plots – correlation coefficient for quantitative data – computational
formula for correlation coefficient – Regression – regression line – least squares
regression line – Standard error of estimate – interpretation of r2 – multiple regression
equations – regression towards the mean

PART A
1. Define Normal curve and its properties.
The normal curve is a theoretical curve defined for a continuous variable, and noted for its
symmetrical bell-shaped form.
Properties of Normal curve:
The normal curve is symmetrical, its lower half is the mirror image of its upper half. Being bell
shaped, the normal curve peaks above a point midway along the horizontal spread and then tapers
off gradually in either direction from the peak(without actually touching the horizontal axis, since,
in theory, the tails of a normal curve extend infinitely far)
2. What is Z-score?
A z score is a unit-free, standardized score that, regardless of the original units of measurement,
indicates how many standard deviations a score is above or below the mean of its distribution.
where X is the original score and μ and σ are the mean and the standard deviation, respectively.
3. What is correlation?
Correlation is a statistical measure that expresses the extent to which two variables are linearly
related. It’s a common tool for describing simple relationships without making a statement about
cause and effect.
The sample correlation coefficient, r, quantifies the strength of the relationship. Correlations are
also tested for statistical significance.
4. Define Scatterplots?
A scatterplot is a graph containing a cluster of dots that represents all pairs of [Link] a little
training, you can use any dot cluster as a preview of a fully measured relationship.
5. What is correlation coefficient?
A correlation coefficient is a number between –1 and 1 that describes the relationship between
pairs of variables. The type of correlation coefficient, designated as r, that describes the linear
relationship between pairs of variables form quantitative data.
6. Define Regression.
A predictive modeling technique that evaluates the relation between dependent (i.e. the target
variable) and independent variables is known as regression analysis. Regression analysis can be
used for forecasting, time series modeling, or finding the relation between the variables and
predict continuous values.
7. Write the types of Regression
[Link] of Regression
Analysis:
1. Simple Linear Regression
2. Multiple Linear Regression
3. Polynomial Regression
4. Logistic Regression
5. Ridge Regression

TEC/CSE/CS3352-FDS/DGD
 6. Lasso Regression
7. Bayesian Linear Regression
8. Decision Tree Regression
9. Random Forest Regression
8. Diffrentiate single and multiple linear
regression?Simple Linear Regression:
Linear regression is the most basic form of regression algorithms in machine learning. The model
consists of a single parameter and a dependent variable has a linear relationship. We denote simple
linear regression by the following equation given below
y = mx + c + e
where m is the slope of the line, c is an intercept, and e represents the error in the model
Multiple Linear Regression:
When the number of independent variables increases, it is called the multiple linear regression
models.
y = b0 + b1x1
9. What is ridge regression?
Ridge Regression is another type of regression in machine learning and is usually used when
there is a high correlation between the parameters. This is because as the correlation increases the
least squareestimates give unbiased values.
10. What is decision tree?
The decision tree as the name suggests works on the principle of conditions. It is efficient and has
strong algorithms used for predictive analysis. It has mainly attributed that include internal nodes,
branches, and a terminal node. Every internal node holds a “test” on an attribute, branches hold the
conclusion of the test and every leaf node means the class label. It is used for both classifications
as well as regression which are both supervised learning algorithms.
11. What is positive correlation?
Positive Correlation – when the values of the two variables move in the same direction so that
an increase/decrease in the value of one variable is followed by an increase/decrease in the
value of the other variable.
12. What is Negative Correlation?
Negative Correlation – when the values of the two variables move in the opposite direction so
that an increase/decrease in the value of one variable is followed by decrease/increase in the
value of the other variable.
13. What is positive relationship?
The first step is to note the tilt or slope, if any, of a dot cluster.
A dot cluster that has a slope from the lower left to the upper right, as in panel A of below
figure reflects a positive relationship.

TEC/CSE/CS3352-FDS/DGD
14. What is Negative relationship?

A dot cluster that has a slope from the upper left to the lower right, as in panel B of below
figure reflects a negative relationship.

15. What is little or No relationship?

A dot cluster that lacks any apparent slope, as in panel C of below figure reflects little or no
relationship.

16. State the formula for correlation.

COMPUTATION FORMULA FOR r
Calculate a value for r by using the following computation formula:

Where the two sum of squares terms in the denominator are defined as

The sum of the products term in the numerator, SPxy\

17. What is predictive error?

TEC/CSE/CS3352-FDS/DGD
Predictive Errors or Prediction error refers to the difference between the predicted values made
by some model and theactual values.

18. What is the STANDARD ERROR OF ESTIMATE ,s y | x

The standard error of the estimate is a measure of the accuracy of predictions. The
regression line is the line that minimizes the sum of squared deviations of prediction (also
called the sum of squares error), and the standard error of the estimate is the square root
of the average squared deviation.
19. What is The Regression Fallac?
The regression fallacy is committed whenever regression toward the mean is
interpreted as a real, ratherthan a chance, effect.
The regression fallacy can be avoided by splitting the subset of extreme observations
into two groups
20. What is regression toward the mean?
Regression toward the mean refers to a tendency for scores, particularly extreme
scores, to shrink toward themean.
In statistics, regression toward the mean (also called reversion to the mean, and
reversion to mediocrity) is a concept that refers to the fact that if one sample of a
random variable is extreme, the next sampling of the samerandom variable is likely to be
closer to its mean.

TEC/CSE/CS3352-FDS/DGD
 Part – B
1. Explain about correlation and Scatter plots in detail.

Correlation
Correlation refers to a process for establishing the relationships between two variables.
You learned a way to get a general idea about whether or not two variables are related, is
to plot them on a “scatter plot”. While there are many measures of association for
variables which are measured at the ordinal or higher level of measurement, correlation is
the most commonly used approach.

Types of Correlation
 Positive Correlation – when the values of the two variables move in the same
direction so that an increase/decrease in the value of one variable is followed by
an increase/decrease in the value of the other variable.
 Negative Correlation – when the values of the two variables move in the
opposite direction so that an increase/decrease in the value of one variable is
followed by decrease/increase in the value of the other variable.
 No Correlation – when there is no linear dependence or no relation between the two
variables.

TEC/CSE/CS3352-FDS/DGD
SCATTERPLOTS
A scatter plot is a graph containing a cluster of dots that represents all pairs of
scores. In other words Scatter plots are the graphs that present the relationship between
two variables in a data-set. It represents datapoints on a two-dimensional plane or on a
Cartesian system.

Construction of scatter plots

 The independent variable or attribute is plotted on the X-axis.
 The dependent variable is plotted on the Y-axis.
2. Explain about Least Square Regression.

REGRESSION
A regression is a statistical technique that relates a dependent variable to one or
more independent (explanatory) variables. A regression model is able to show whether
changes observed in the dependent variable are associated with changes in one or more of
the explanatory variables.
Regression captures the correlation between variables observed in a data set, and
quantifies whether those correlations are statistically significant or not.

A Regression Line
a regression line is a line that best describes the behaviour of a set of data. In other
words, it’s a line that bestfits the trend of a given data.

The purpose of the line is to describe

the interrelation of a dependent
variable (Y variable) with one or many
independent variables (X variable). By
using the equation obtained from the
regression line an analyst can forecast
future behaviours of the dependent
variable by inputting different values
for the independent ones.

Types of regression
The two basic types of regression are
 Simple linear regression
Simple linear regression uses one
independent variable toexplain or
predict the outcome of the dependent
variable Y
 Multiple linear regression
Multiple linear regressions use
two or more independentvariables to
predict the outcome

Predictive Errors
Prediction error refers to the difference between the predicted values made by
some model and theactual values.

TEC/CSE/CS3352-FDS/DGD
LEAST SQUARES REGRESSION LINE
The placement of the regression line minimizes not the total predictive error but
the total squared predictive error, that is, the total for all squared predictive errors. When
located in this fashion, the regression line is often referred to as the least squares
regression line.
The Least Squares Regression Line is the line that minimizes the sum of the
residuals squared. The residual is the vertical distance between the observed point and the
predicted point, and it is calculated by subtracting ˆy from y.

Formula

y’ = bx+a b – slope , a – y intercept

b= N Σ(xy) − Σx Σy
N Σ(x2) − (Σx)2
b = Σy − m Σx
N
Example

"x" "y"
2 4
3 5
5 7
7 10
9 15

Step 1: For each (x,y) calculate x2 and xy:

x y x2 xy

2 4 4 8
3 5 9 15
5 7 25 35

TEC/CSE/CS3352-FDS/DGD
 7 10 49 70
9 15 81 135
Step 2: Sum x, y, x2 and xy (gives us Σx, Σy, Σx2 and Σxy):
Σx: 26 Σy: 41 Σx2: 168 Σxy: 263
Step 3:
Calculate
Slope b
b = N Σ(xy) − Σx Σy N Σ(x2) − (Σx)2
= 5 x 263 − 26 x 41
5 x 168 − 262

= 1315 − 1066
840 – 676
=
249
164
b = 1.5183.

Step 4: Calculate Intercept aa = Σy − b Σx

N
= 41 −
1.5183 x
26
5
a=
0.3049.

Step 5: y’ = bx+a
y’ = 1.518x + 0.305

x y y = 1.518x + 0.305 error

2 4 3.34 −0.66
3 5 4.86 −0.14
5 7 7.89 0.89
7 10 10.93 0.93
9 15 13.97 −1.03

To predict the y value we can

assume any value for [Link]
x = 8.
Then y = 1.518 x 8 + 0.305
= 12.45
 Use each pair of scores to locate a dot within the scatter plot

TEC/CSE/CS3352-FDS/DGD
Positive, Negative, or Little or No Relationship?

The first step is to note the tilt or slope, if any, of a dot cluster.
A dot cluster that has a slope from the lower left to the upper right, as in panel A of below
figure reflects a
positive relationship.

A dot cluster that has a slope from the upper left to the lower right, as in panel B of below
figure reflects a
negative relationship.

A dot cluster that lacks any apparent slope, as in panel C of below figure reflects little or no
relationship.

Perfect Relationship
A dot cluster that equals (rather than merely approximates) a straight line reflects a
perfect relationship between two variables.

Curvilinear Relationship
The previous discussion assumes that a dot cluster approximates a straight line and,
therefore, reflects a linear relationship. But this is not always the case. Sometimes a dot
cluster approximates a bent or curved line, as in below figure, and therefore reflects a
curvilinear relationship.

TEC/CSE/CS3352-FDS/DGD
3. Explain about the Correlation Coefficient For Quantitative Data : r.
The correlation coefficient, r, is a summary measure that describes the extent of
the statisticalrelationship between two interval or ratio level variables.

Properties of r
 The correlation coefficient is scaled so that it is always between -1 and +1.
 When r is close to 0 this means that there is little relationship between the
variables and the farther away from 0 r is, in either the positive or negative
direction, the greater the relationship between the two variables.
 The sign of r indicates the type of linear relationship, whether positive or negative.
 The numerical value of r, without regard to sign, indicates the strength of the linear
relationship.
 A number with a plus sign (or no sign) indicates a positive relationship, and a
number with a minus sign indicates a negative relationship

COMPUTATION FORMULA FOR r

Calculate a value for r by using the following computation formula:

Where the two sum of squares terms in the denominator are defined as

The sum of the products term in the numerator, SPxy, is defined in below formula

Or the formula is written as

TEC/CSE/CS3352-FDS/DGD
 Where n = Number of Information
Σx = Total of the
First Variable
Value Σy = Total
of the Second
Variable Value
Σxy = Sum of the Product of
first & Second ValueΣx2 =
Sum of the Squares of the
First Value
Σy2 = Sum of the Squares of the Second Value

4. Explain about The Standard Error Of Estimate ,s y | x

The standard error of the estimate is a measure of the accuracy of predictions. The
regression line is the line that minimizes the sum of squared deviations of prediction (also
called the sum of squares error), and the standard error of the estimate is the square root
of the average squared deviation.
The standard error of estimate and symbolized as s y | x, this estimate of
predictive error complies with the general format for any sample standard deviation, that

TEC/CSE/CS3352-FDS/DGD
 is, the square root of a sum of squares term dividedby its degrees of freedom.

Fig. Predictive errors for five friends

Example
Calculate the standard error of estimate for the given X and Y values. X = 1,2,3,4,5 Y=2,4,5,4,5
Solution
Create five columns labeled x, y, y’, y – y’, ( y – y’)2 and N=5

TEC/CSE/CS3352-FDS/DGD
 x y x2 xy Y’= y-y’ ( y – y’)2
bx+a
1 2 1 2 2.8 -0.8 0.64
2 4 4 8 3.4 0.6 0.36
3 5 9 15 4.0 1 1
4 4 16 16 4.6 -0.6 0.36
5 5 25 25 5.2 -0.2 0.04
Σx:15 Σy:20 Σx2:55 Σxy:66 Σ( y – y’)2
= 2.4

Note: for finding b value we have to find xy and x2, so add xy and x2 column in table

b = N Σ(xy) − Σx Σy N Σ(x2) − (Σx)2

b=5(66)-15x20 5(55)-(15)2

= 330 – 300
275-225

b= 30/50 = 0.6

a = Σy − b Σx
N
= 20 – (0.6 x 15)
5
= 20 – 11
5
a= 9/5 = 2.2

SSy/x = √((y-y’)2 / n-2)

=√(2.4/3)

SSy/x = 0.894

5. Explain about the interpretation of r 2

R-Squared (R² or the coefficient of determination) is a statistical measure in a
regression model that determines the proportion of variance in the dependent variable that
can be explained by the independent variable. In other words, r-squared shows how well
the data fit the regression model (the goodness of fit).
R-squared can take any values between 0 to 1. Although the statistical measure
provides some useful insights regarding the regression model, the user should not rely
only on the measure in the assessment of a statistical model.

In addition, it does not indicate the correctness of the regression model. Therefore,

TEC/CSE/CS3352-FDS/DGD
 the user should always draw conclusions about the model by analyzing r-squared together
with the other variables in astatistical model.
The most common interpretation of r-squared is how well the regression model explains
observed data.

MULTIPLE REGRESSION EQUATIONS

Multiple regression is a statistical technique applied on datasets dedicated to draw
out a relationship between one response or dependent variable and multiple independent
variables.
Multiple regression works by considering the values of the available multiple
independent variables and predicting the value of one dependent variable.

Example:
A researcher decides to study students’ performance from a school over a period of time.
He observed that as the lectures proceed to operate online, the performance of students
started to decline as well. The parameters for the dependent variable “decrease in
performance” are various independent variables like “lack of attention, more internet
addiction, neglecting studies” and much more.

Formula to find
multiple regression
y = b1x1 + b2x2 + … bnxn + a

REGRESSION TOWARD THE MEAN

Regression toward the mean refers to a tendency for scores, particularly extreme
scores, to shrink toward themean.
In statistics, regression toward the mean (also called reversion to the mean, and
reversion to mediocrity) is a concept that refers to the fact that if one sample of a
random variable is extreme, the next sampling of the samerandom variable is likely to be
closer to its mean.

Example
A military commander has two units return, one with 20% casualties and another with
50% casualties. He praises the first and berates the second. The next time, the two units
return with the opposite results. From this experience, he “learns” that praise weakens
performance and berating increases performance.

The Regression Fallacy

The regression fallacy is committed whenever regression toward the mean is
interpreted as a real, ratherthan a chance, effect.
The regression fallacy can be avoided by splitting the subset of extreme observations
into two groups

TEC/CSE/CS3352-FDS/DGD
TEC/CSE/CS3352-FDS/DGD
 CS3352 - FOUNDATION
UNIT OF
– I DATA SCIENCE
Question Bank
INTRODUCTION
UNIT IV - PYTHON LIBRARIES FOR DATA WRANGLING

Basics of Numpy arrays –aggregations –computations on arrays –comparisons, masks, boolean logic –
fancyindexing – structured arrays – Data manipulation with Pandas – data indexing and selection –
operating on data– missing data – Hierarchical indexing – combining datasets – aggregation and grouping
– pivot tables
PART A

1. What is NumPy in Python used for?

NumPy is a Python library used for working with arrays.
2. Write a python program create an array?
Import numpy as [Link]([1,4,2,5,3])
OUTPUT: array([1,4,2,5,3])
3. Write the output of the following numpy code:
i. [Link]([3.14, 4, 2, 3])
ii. [Link]([1, 2, 3, 4], dtype='float32')
vii. :
[Link]([3.14, 4. , 2. , 3. ])
ii. array([1., 2., 3., 4.], dtype=float32)
4. Define Series Object.
A Pandas Series is a one-dimensional array of indexed data. It can be created from a list or [Link]:
data = [Link]([0.25, 0.5, 0.75, 1.0])print(data )
The output will be displayed as
0 0.25
1 0.50
2 0.75
3 1.00
dtype: float64
5. What is Data frame?
A DataFrame is an analog of a two-dimensional array with both flexible row indices and flexiblecolumn
names. DataFrame as a sequence of aligned Series objects.
Example:
states = [Link]({'population': population, 'area': area})print(states)
Output:
area population
California 423967 38332521
Florida 170312 19552860
Illinois 149995 12882135
New York 141297 19651127
Texas 695662 26448193
6. How a pandas dataframe can be constructed?

TEC/CSE/CS3352-FDS/DGD
i) From a single Series object
ii) From a list of dicts
iii) From a dictionary of Series objects
iv) From a two-dimensional NumPy array
v) From a NumPy structured array
7. What are indexers?
Pandas provides some special indexer attributes that explicitly expose indexing schemes. They are loc,
iloc, and ix.
loc attribute - allows indexing and slicing that always references the explicit index.
iloc attribute - allows indexing and slicing that always references the implicit Python-style [Link] - is a
hybrid of the two, and for Series objects is equivalent to standard []-based indexing.
8. How missing data can be handled in python?
None, a Python singleton object that is often used for missing data in Python [Link]:
import numpy as npimport pandas as pd
val = [Link]([1, None, 3, 4])(val)
Output: array([1, None, 3, 4], dtype=object)
The other missing data representation, NaN ( Not a Number), is a special floating-point valuerecognized
by all systems that use the standard IEEE floating-point representation:Example
val = [Link]([1, [Link], 3, 4])print( [Link])
Output: dtype('float64')
9. How the operations can be performed on null values in pandas data structure?
There are several useful methods for detecting, removing, and replacing null values in Pandas data
structures.
They are:
isnull() - Generate a Boolean mask indicating missing valuesnotnull() - Opposite of isnull()
dropna() - Return a filtered version of the data
fillna() - Return a copy of the data with missing values filled or imputed
10. Define Hierarchical Indexing.
Hierarchical indexing also known as multi-indexing is used to incorporate multiple index levels within a
single index. In this way, higher-dimensional data can be compactly represented within the familiar one-
dimensional Series and two-dimensional DataFrame objects.
11. What is pivot table?
The pivot table takes simple column-wise data as input, and groups the entries into a two- dimensional
table that provides a multidimensional summarization of the data.
12. What are the attributes of NumPy Array?
ndim (the number of dimensions),
shape (the size of each dimension)
size (the total size of the array)
13. What is the purpose of NumPy UFuncs
NumPy provides a convenient interface into just this kind of statically typed, compiled routine. This is
known as a vectorized [Link] operations in NumPy are implemented via ufuncs, whose
main purpose is to quickly execute repeated operations on values in NumPy arrays. Ufuncs are extremely
flexible.
14. State the Rules of Broadcasting
Broadcasting in NumPy follows a strict set of rules to determine the interaction between the two arrays.

TEC/CSE/CS3352-FDS/DGD
Rule 1: If the two arrays differ in their number of dimensions, the shape of the one with fewer dimensions
is padded with ones on its leading (left) side.
Rule 2: If the shape of the two arrays does not match in any dimension, the array with shape equal to 1 in
thatdimension is stretched to match the other shape.
Rule 3: If in any dimension the sizes disagree and neither is equal to 1, an error is raised.
15. What is Fancy Indexing?
Fancy indexing is like the simple indexing we‘ve already seen, but we pass arrays of indices in place of
single scalars. This allows us to very quickly access and modify complicated subsets of an array‘s values.
16. What is Pandas?
Pandas is a newer package built on top of NumPy, and provides an efficient implementation of a
DataFrame. DataFrames are essentially multidimensional arrays with attached row and column labels, and
oftenwith heterogeneous types and/or missing data.
17. What is Pandas Objects?
Pandas objects can be thought of as enhanced versions of NumPy structured arrays in which the
rows andcolumns are identified with labels rather than simple integer indices.
18. State how to handle missing data in python.
None: Pythonic missing data
The first sentinel value used by Pandas is None, a Python singleton object that is often used for missing
data in Python code. Because None is a Python object, it cannot be used in any arbitrary NumPy/Pandas
array, but onlyin arrays with data type 'object' (i.e., arrays of Python objects)
This dtype=object means that the best common type representation NumPy could infer for the contents of
the array is that they are Python objects.
19. State how to handle missing numerical data in python.
NaN: Missing numerical data
NaN is a special floating-point value recognized by all systems that use the standard IEEE floating-point
representation.
vals2 = [Link]([1, [Link], 3, 4]) [Link]
dtype('float64')
20. Listing of Pandas aggregation methodsAggregation
Description
count() Total number of items
first(), last() First and last itemmean(), median()Mean and median
min(), max() Minimum and maximum
std(), var() Standard deviation and variance
mad() Mean absolute deviation
prod() Product of all items
sum() Sum of all items
21. What is Filtering?
A filtering operation allows you to drop data based on the group properties. For example, we might
want tokeep all groups in which the standard deviation is larger than some critical value.
The filter() function should return a Boolean value specifying whether the group passes the filtering.
22. What is Transformation.
While aggregation must return a reduced version of the data, transformation can return some transformed
version of the full data to recombine. For such a transformation, the output is the same shape as the input.
A common example is to center the data by subtracting the group-wise mean:

TEC/CSE/CS3352-FDS/DGD
 PART B
1. Briefly explain the basics of numpy arrays with example
NumPy (short for Numerical Python) provides an efficient interface to store and operate on dense data
[Link] arrays are like Python‘s built-in list type, but NumPy arrays provide much more efficient
storage and data operations as the arrays grow larger in size.
Attributes of arrays
Determining the size, shape, memory consumption, and data types of arrays
Indexing of arrays
Getting and setting the value of individual array elements
Slicing of arrays
Getting and setting smaller subarrays within a larger array
Reshaping of arrays
Changing the shape of a given array
Joining and splitting of arrays
Combining multiple arrays into one, and splitting one array into many
NumPy Array Attributes
 ndim (the number of dimensions),
 shape (the size of each dimension)
 size (the total size of the array)
Example
[Link](0) # seed for reproducibility
x1 = [Link](10, size=6) # One-dimensional array
x2 = [Link](10, size=(3, 4)) # Two-dimensional array
x3 = [Link](10, size=(3, 4, 5)) # Three-dimensional array
print("x3 ndim: ", [Link])print("x3 shape:", [Link]) print("x3 size: ", [Link])
print("dtype:", [Link])
print("itemsize:", [Link], "bytes")print("nbytes:", [Link], "bytes")
Array Indexing:
Accessing Single Elements
Accessing Single Elements
Indexing in NumPy will feel quite familiar like list indexing,
In a one-dimensional array, you can access the ith value (counting from zero) by specifying the desired
index in square brackets, just as with Python lists
To index from the end of the array, you can use negative indices
In a multidimensional array, you access items using a comma-separated tuple of indices
Unlike Python lists, NumPy arrays have a fixed type. This means, for example, that if you attempt to insert
a floating-point value to an integer array, the value will be silently truncated.
Array Slicing: Accessing Subarrays
Just as we can use square brackets to access individual array elements, we can also use them to access
subarrayswith the slice notation, marked by the colon (:) character.
The NumPy slicing syntax follows that of the standard Python list; to access a slice ofan array x, use this:
x[start:stop:step]
start – starting array index
stop – array index to stop ( last value will not be considered)step – terms has to be printed from start to

TEC/CSE/CS3352-FDS/DGD
stop
Default to the values start=0, stop=size of dimension, step=1.
Example
x = [Link](10)x
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
x[:5] # prints first five elements
array([0, 1, 2, 3, 4])
x[5:] # elements after index 5
array([5, 6, 7, 8, 9])

x[4:7] # middle subarray(from 4th index to 6th index)

array([4, 5, 6])

While using negative indices the defaults for start and stop are swapped. This becomes a convenient way
toreverse an array
x[::-1] # all elements, reversed
array([9, 8, 7, 6, 5, 4, 3, 2, 1, 0])

x[5::-2] # reversed every other from index 5

array([5, 3, 1])
Multidimensional sub arrays
Multidimensional slices work in the same way, with multiple slices separated by [Link] example:
x2
array([[12, 5, 2, 4],
[ 7, 6, 8, 8],
[ 1, 6, 7, 7]])
x2[:2, :3] # two rows, three columns array([[12, 5, 2],
[ 7, 6, 8]])

x2[:3, ::2] # all rows, every other column(every second column) array([[12, 2],
[ 7, 8],
[ 1, 7]])
Finally, sub array dimensions can even be reversed together
x2[::-1, ::-1]
array([[ 7, 7, 6, 1],
[ 8, 8, 6, 7],
[ 4, 2, 5, 12]])
Reshaping of Arrays
The most flexible way of doing this is with the reshape() method. For example, if you want to put the
numbers1 through 9 in a 3×3 grid, you can do the following
grid = [Link](1, 10).reshape((3, 3)) print(grid)
[[1 2 3]
[4 5 6]
[7 8 9]]

TEC/CSE/CS3352-FDS/DGD
Array Concatenation and Splitting
Concatenation of arrays
Concatenation, or joining of two arrays in NumPy, is primarily accomplished through the routines
[Link], [Link], and [Link]. [Link] takes a tuple or list of arrays as its first
argument.x = [Link]([1, 2, 3])
y = [Link]([3, 2, 1]) [Link]([x, y])

array([1, 2, 3, 3, 2, 1])
You can also concatenate more than two arrays at once
z = [99, 99, 99]
print([Link]([x, y, z]))[ 1 2 3 3 2 1 99 99 99]
[Link] can also be used for two-dimensional arrays
grid = [Link]([[1, 2, 3],
[4, 5, 6]])
[Link]([grid, grid])
array([[1, 2, 3],
[4, 5, 6],
[1, 2, 3],
[4, 5, 6]])
Concatenate along the second axis (zero-indexed)
[Link]([grid, grid], axis=1)
array([[1, 2, 3, 1, 2, 3],
[4, 5, 6, 4, 5, 6]])
[Link] (vertical stack) functions
x = [Link]([1, 2, 3])
grid = [Link]([[9, 8, 7],
[6, 5, 4]])
[Link]([x, grid])
array([[1, 2, 3],
[9, 8, 7],
[6, 5, 4]])
[Link] (horizontal stack) functions
y = [Link]([[99],
[99]])
[Link]([grid, y])
array([[ 9, 8, 7, 99],
[ 6, 5, 4, 99]])
Splitting of arrays
The opposite of concatenation is splitting, which is implemented by the functions [Link], [Link], and
[Link]. For each of these, we can pass a list of indices giving the split points
x = [1, 2, 3, 99, 99, 3, 2, 1]
x1, x2, x3 = [Link](x, [3, 5]) print(x1, x2, x3)
[1 2 3] [99 99] [3 2 1]
Notice that N split points lead to N + 1 subarrays. The related functions [Link] and [Link] are similar
grid = [Link](16).reshape((4, 4)) grid

TEC/CSE/CS3352-FDS/DGD
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
upper, lower = [Link](grid, [2])
print(upper)print(lower)
[[0 1 2 3]
[4 5 6 7]]
[[ 8 9 10 11]
[12 13 14 15]]
left, right = [Link](grid, [2])
print(left) print(right)
[[ 0 1]
[ 4 5]
[ 8 9]
[12 13]]
[[ 2 3]
[ 6 7]
[10 11]
[14 15]]

2. Describe about fancy indexing with example

Fancy indexing is like the simple indexing we‘ve already seen, but we pass arrays of indices in place of
single scalars. This allows us to very quickly access and modify complicated subsets of an array‘s values.
Exploring Fancy Indexing
Fancy indexing is conceptually simple: it means passing an array of indices to access multiple array
elements atonce.
Types of fancy indexing.
 Indexing / accessing more values
 Array of indices
 In multi dimensional
 Standard indexing
Example
import numpy as np
rand = [Link](42)x = [Link](100, size=10) print(x)
[51 92 14 71 60 20 82 86 74 74]
Indexing / accessing more values
Suppose we want to access three different elements. We could do it like this:[x[3], x[7], x[2]]
[71, 86, 14]
Array of indices
We can pass a single list or array of indices to obtain the same [Link] = [3, 7, 4]
x[ind]
array([71, 86, 60])
In multi dimensional
Fancy indexing also works in multiple dimensions. Consider the following array.

TEC/CSE/CS3352-FDS/DGD
X = [Link](12).reshape((3, 4))X
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
Standard indexing
Like with standard indexing, the first index refers to the row, and the second to the column. row =
[Link]([0, 1, 2])
col = [Link]([2, 1, 3])
X[row, col] array ([ 2, 5, 11])
Combined Indexing
For even more powerful operations, fancy indexing can be combined with the other indexing schemes
we‘veseen.
Example array
print(X)
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
Combine fancy and simple indices
X[2, [2, 0, 1]]
array([10, 8, 9])
Combine fancy indexing with slicing
X[1:, [2, 0, 1]]
array([[ 6, 4, 5],
[10, 8, 9]])
Combine fancy indexing with masking mask = [Link]([1, 0, 1, 0], dtype=bool) X[row[:, [Link]],
mask]
array([[ 0, 2],
[ 4, 6],
[ 8, 10]])
Modifying Values with Fancy Indexing
Just as fancy indexing can be used to access parts of an array, it can also be used to modify parts of an
[Link] some value in an array
Modify particular element by index
For example, imagine we have an array of indices and we‘d like to set the corresponding items in an array
tosome value.
x = [Link](10)
i = [Link]([2, 1, 8, 4])
x[i] = 99
print(x)
[ 0 99 99 3 99 5 6 7 99 9]
Using assignment operator
We can use any assignment-type operator for this. For example
x[i] -= 10
print(x)
x = [Link](10) [Link](x, i, 1) print(x)

TEC/CSE/CS3352-FDS/DGD
 [ 0 89 89 3 89 5 6 7 89 9]
Using at()
Use the at() method of ufuncs for other behavior of modifications.
[ 0. 0. 1. 2. 3. 0. 0. 0. 0. 0.]

3. Explain structured data in numpy array.

This section demonstrates the use of NumPy‘s structured arrays and record arrays, which provide
efficientstorage for compound, heterogeneous data.
NumPy data types
Character Description Example
'b' Byte [Link]('b')
'i' Signed integer [Link]('i4') == np.int32
'u' Unsigned integer [Link]('u1') == np.uint8
'f' Floating point [Link]('f8') == np.int64
'c' Complex floating point [Link]('c16') == np.complex128
'S', 'a' string [Link]('S5')
'U' Unicode string [Link]('U') == np.str_'V' Raw data (void)
[Link]('V') == [Link]
Consider if we have several categories of data on a number of people (say, name, age, and weight), and
we‘d like to store these values for use in a Python program. It would be possible to store these in three
separate arrays.
name = ['Alice', 'Bob', 'Cathy', 'Doug'] age = [25, 45, 37, 19]
weight = [55.0, 85.5, 68.0, 61.5]

Creating structured array

NumPy can handle this through structured arrays, which are arrays with compound data types. create a
structured array using a compound data type specification as follows.
data = [Link](4, dtype={'names':('name', 'age', 'weight'),
'formats':('U10', 'i4', 'f8')})
print([Link])
[('name', '<U10'), ('age', '<i4'), ('weight', '<f8')] U10 - Unicode string of maximum length 10 i4 - 4-byte
(i.e., 32 bit) integer
f8 - 8-byte (i.e., 64 bit) float
Now we can fill the array with our lists of values
data['name'] = name data['age'] = age data['weight'] = weight print(data)
[('Alice', 25, 55.0) ('Bob', 45, 85.5) ('Cathy', 37, 68.0)('Doug', 19, 61.5)]
Refer values through index or name
The handy thing with structured arrays is that you can now refer to values either by index or by name.
i. data['name']# by name
array(['Alice', 'Bob', 'Cathy', 'Doug'],dtype='<U10')
ii. data[0]# by index
('Alice', 25, 55.0)
Using Boolean masking
This allows to do some more sophisticated operations such as filtering on any fields.
data[data['age'] < 30]['name']

TEC/CSE/CS3352-FDS/DGD
array(['Alice', 'Doug'],dtype='<U10')
Creating Structured ArraysDictionary method
[Link]({'names':('name', 'age', 'weight'),
'formats':('U10', 'i4', 'f8')}) dtype([('name', '<U10'), ('age', '<i4'), ('weight', '<f8')])
Numerical types can be specified with Python types
[Link]({'names':('name', 'age', 'weight'),
'formats':((np.str_, 10), int, np.float32)}) dtype([('name', '<U10'), ('age', '<i8'), ('weight', '<f4')])
List of tuples
[Link]([('name', 'S10'), ('age', 'i4'), ('weight', 'f8')])

dtype([('name', 'S10'), ('age', '<i4'), ('weight', '<f8')])

Specify the types alone
[Link]('S10,i4,f8')
dtype([('f0', 'S10'), ('f1', '<i4'), ('f2', '<f8')])

4. What is universal function? Explain clearly each function with example.

Introducing UFuncs
NumPy provides a convenient interface into just this kind of statically typed, compiled routine. This is
knownas a vectorized operation.
Vectorized operations in NumPy are implemented via ufuncs, whose main purpose is to quickly execute
repeated operations on values in NumPy arrays. Ufuncs are extremely flexible—before we saw an
operation between a scalar and an array, but we can also operate between two arrays
Exploring NumPy’s UFuncs
Ufuncs exist in two flavors: unary ufuncs, which operate on a single input, and binary ufuncs, which
operate on two inputs. We‘ll see examples of both these types of functions here.
Array arithmetic
NumPy‘s ufuncs make use of Python‘s native arithmetic operators. The standard addition, subtraction,
multiplication, and division can all be used.
x = [Link](4) print("x =", x) print("x + 5 =", x + 5) print("x - 5 =", x - 5) print("x * 2 =", x * 2)

Ope Equivalent ufunc Description

rato
r
+ [Link] Addition (e.g., 1 + 1 = 2)
- [Link] Subtraction (e.g., 3 - 2 = 1)
- [Link] Unary negation (e.g., -2)
* [Link] Multiplication (e.g., 2 * 3 = 6)
/ [Link] Division (e.g., 3 / 2 = 1.5)
// np.floor_divide Floor division (e.g., 3 // 2 = 1)
** [Link] Exponentiation (e.g., 2 ** 3 = 8)
% [Link] Modulus/remainder (e.g., 9 % 4 = 1)

Absolute value
Just as NumPy understands Python‘s built-in arithmetic operators, it also understands Python‘s built-in

TEC/CSE/CS3352-FDS/DGD
absolutevalue function.
[Link]()
[Link]()
x = [Link]([-2, -1, 0, 1, 2]) abs(x)
array([2, 1, 0, 1, 2])
The corresponding NumPy ufunc is [Link], which is also available under the alias [Link]
[Link](x) array([2, 1, 0, 1, 2])
[Link](x)
array([2, 1, 0, 1, 2])
Trigonometric functions
NumPy provides a large number of useful ufuncs, and some of the most useful for the data scientist
are thetrigonometric functions.
 [Link]()
 [Link]()
 [Link]()
 inverse trigonometric functions
 [Link]()
 [Link]()
 [Link]()
Defining an array of angles: theta = [Link](0, [Link], 3)
Compute some trigonometric functions like
print("theta = ", theta) print("sin(theta) = ", [Link](theta)) print("cos(theta) = ", [Link](theta))
print("tan(theta) = ", [Link](theta))
Exponents and logarithms
Another common type of operation available in a NumPy ufunc are the exponentials.
 [Link](x) – calculate exponent of all elements in the input array ie ex ( e=2.7182)
 np.exp2(x) – calculate 2**x for all x being the array elements
 [Link](x,y) – calculates the power as xy
x = [1, 2, 3]
print("x =", x)
print("e^x =", [Link](x))
print("2^x =", np.exp2(x))
print("3^x =", [Link](3, x))

The inverse of the exponentials, the logarithms, are also available. The basic [Link] gives the natural
logarithm;if you prefer to compute the base-2 logarithm or the base-10 logarithm as .
 [Link](x) - is a mathematical function that helps user to calculate Natural logarithm of x where x
belongsto all the input array elements
 np.log2(x) - to calculate Base-2 logarithm of x
 np.log10(x) - to calculate Base-10 logarithm of x

x = [1, 2, 4, 10]
print("x =", x)
print("ln(x) =", [Link](x))

TEC/CSE/CS3352-FDS/DGD
print("log2(x) =", np.log2(x))
print("log10(x) =", np.log10(x))

Specialized ufuncs
NumPy has many more ufuncs available like
 Hyperbolic trig functions,
 Bitwise arithmetic,
 Comparison operators,
 Conversions from radians to degrees,
 Rounding and remainders, and much more
More specialized and obscure ufuncs is the submodule [Link]. If you want to compute some
obscuremathematical function on your data, chances are it is implemented in [Link].
Gamma function
Advanced Ufunc FeaturesSpecifying output
Rather than creating a temporary array, you can use this to write computation results directly to the
memorylocation where you‘d like them to be. For all ufuncs, you can do this using the out argument of the
function.
x = [Link](5)y = [Link](5)
[Link](x, 10, out=y)
print(y)
[ 0. 10. 20. 30. 40.]
Aggregates
To reduce an array with a particular operation, we can use the reduce method of any ufunc. A reduce
repeatedlyapplies a given operation to the elements of an array until only a single result remains.
x = [Link](1, 6)[Link](x)
Similarly, calling reduce on the multiply ufunc results in the product of all array elements
[Link](x)120
If we‘d like to store all the intermediate results of the computation, we can instead useAccumulate
[Link](x) array([ 1, 3, 6, 10, 15])
Outer products
ufunc can compute the output of all pairs of two different inputs using the outer method. This allows you,
in oneline, to do things like create a multiplication table.
x = [Link](1, 6) [Link](x, x)
array([[ 1, 2, 3, 4, 5],
[ 2, 4, 6, 8, 10],
[ 3, 6, 9, 12, 15],
[ 4, 8, 12, 16, 20],
[ 5, 10, 15, 20, 25]])

5. Explain aggregate functions with example

Aggregations: Min, Max, and Everything in Between
Minimum and Maximum
Python has built-in min and max functions, used to find the minimum value and maximumvalue of
any given array.
For min, max, sum, and several other NumPy aggregates, a shorter syntax is to use methods ofthe array

TEC/CSE/CS3352-FDS/DGD
object itself.
 [Link]() – finds the minimum (smallest) value in the array
 [Link]() – finds the maximum (largest) value in the arrayExample
x=[1,2,3,4]
[Link](x)1
[Link](x)4
Multidimensional aggregates
One common type of aggregation operation is an aggregate along a row or column.
By default, each NumPy aggregation function will return the aggregate over the entire array. ie. If we
use [Link]() it will calculates the sum of all elements of the array.

Example
m = [Link]((3, 4))
print(M)

[[ 0.8967576 0.03783739 0.75952519 0.06682827]

[ 0.8354065 0.99196818 0.19544769 0.43447084]
[ 0.66859307 0.15038721 0.37911423 0.6687194 ]]
[Link]() 6.0850555667307118

Aggregation functions take an additional argument specifying the axis along which the aggregate is
computed. The axis normally takes either 0 or 1. if the axis = 0 then it runs along with columns, if axis
=1 it runs alongwith rows.

Example
We can find the minimum value within each column by specifying axis=0

[Link](axis=0)
array([ 0.66859307, 0.03783739, 0.19544769, 0.06682827])

Similarly, we can find the maximum value within each [Link](axis=1)

array([ 0.8967576 , 0.99196818, 0.6687194 ])

Other aggregation functions

NumPy provides many other aggregation functions most aggregates have a NaN-safe counterpart that
computes the result while ignoring missing values, which are marked by the special IEEE floating-point
NaN value.
Function Name NaN-safe Version Description
[Link] [Link] Compute sum of elements
[Link] [Link] Compute product of elements
[Link] [Link] Compute median of elements
[Link] [Link] Compute standard deviation
[Link] [Link] Compute variance
[Link] [Link] Find minimum value
[Link] [Link] Find maximum value

TEC/CSE/CS3352-FDS/DGD
 [Link] [Link] Find index of minimum value
[Link] [Link] Find index of maximum value
[Link] [Link] Compute median of elements
[Link] [Link] Compute rank-based statistics of elements
[Link] N/A Evaluate whether any elements are true
[Link] N/A Evaluate whether all elements are true

6. What is broadcasting and explain the rules with examples

Broadcasting is simply a set of rules for applying binary ufuncs (addition, subtraction, multiplication,
etc.) on arrays of different sizes.
For arrays of the same size, binary operations are performed on an element-by-element basis.a =
[Link]([0, 1, 2])
b = [Link]([5, 5, 5])a + b

array([5, 6, 7])
Broadcasting allows these types of binary operations to be performed on arrays of different sizes. a + 5
array([5, 6, 7])
We can think of this as an operation that stretches or duplicates the value 5 into the array [5, 5, 5], and
adds theresults. The advantage of NumPy‘s broadcasting is that this duplication of values does not actually
take place.
We can similarly extend this to arrays of higher dimension. Observe the result when we add a one-
dimensionalarray to a two-dimensional array.
Example
M = [Link]((3, 3))M
array([ [ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.]])
M+a
array([[ 1., 2., 3.],
[ 1., 2., 3.],
[ 1., 2., 3.]])
Here the one-dimensional array a is stretched, or broadcast, across the second dimension in order to
match theshape of M.
Just as before we stretched or broadcasted one value to match the shape of the other, here we‘ve
stretched botha and b to match a common shape, and the result is a two dimensional array.

TEC/CSE/CS3352-FDS/DGD
The light boxes represent the broadcasted values: again, this extra memory is not actually allocated in the
courseof the operation, but it can be useful conceptually to imagine that it is.
Rules of Broadcasting
Broadcasting in NumPy follows a strict set of rules to determine the interaction between the two arrays.
• Rule 1: If the two arrays differ in their number of dimensions, the shape of the one with fewer
dimensionsis padded with ones on its leading (left) side.
• Rule 2: If the shape of the two arrays does not match in any dimension, the array with shape equal
to 1 in thatdimension is stretched to match the other shape.
• Rule 3: If in any dimension the sizes disagree and neither is equal to 1, an error is raised.
Broadcasting example 1
Let‘s look at adding a two-dimensional array to a one-dimensional array:
M = [Link]((2, 3))a = [Link](3)
Let‘s consider an operation on these two arrays. The shapes of the arrays are:[Link] = (2, 3)
[Link] = (3,)
We see by rule 1 that the array a has fewer dimensions, so we pad it on the left with ones:[Link] -> (2,
3)
[Link] -> (1, 3)
By rule 2, we now see that the first dimension disagrees, so we stretch this dimension to match: [Link] -
> (2, 3)
[Link] -> (2, 3)

TEC/CSE/CS3352-FDS/DGD
The shapes match, and we see that the final shape will be (2, 3):
M+a
array([[ 1., 2., 3.],
[ 1., 2., 3.]])
Broadcasting example 2
Let‘s take a look at an example where both arrays need to be broadcast:a = [Link](3).reshape((3, 1))
b = [Link](3)
Again, we‘ll start by writing out the shape of the arrays:[Link] = (3, 1)
[Link] = (3,)
Rule 1 says we must pad the shape of b with ones:[Link] -> (3, 1)
[Link] -> (1, 3)
And rule 2 tells us that we upgrade each of these ones to match the corresponding size of the other array:
[Link] -> (3, 3)
[Link] -> (3, 3)
Because the result matches, these shapes are compatible. We can see this here:a + b
array([[0, 1, 2],
[1, 2, 3],
[2, 3, 4]])

7. Explain data objects in pandas.

Pandas is a newer package built on top of NumPy, and provides an efficient implementation of a
DataFrame. DataFrames are essentially multidimensional arrays with attached row and column labels, and
oftenwith heterogeneous types and/or missing data.
Pandas, and in particular its Series and DataFrame objects, builds on the NumPy array structure and
provides efficient access to these sorts of ―data munging‖ tasks that occupy much of a data scientist‘s
time.
Here we will focus on the mechanics of using Series, DataFrame, and related structures effectively.

Introducing Pandas Objects

Pandas objects can be thought of as enhanced versions of NumPy structured arrays in which the
rows andcolumns are identified with labels rather than simple integer indices.
Pandas provide a host of useful tools, methods, and functionality on top of the basic data [Link]
fundamental Pandas data structures: the Series, DataFrame, and Index
The Pandas Series Object
A Pandas Series is a one-dimensional array of indexed data. It can be created from a list or array as
follows:
data = [Link]([0.25, 0.5, 0.75, 1.0]) data
0 0.25
1 0.50
2 0.75
3 1.00
dtype: float64
 Finding values
The values are simply a familiar NumPy array
[Link]

TEC/CSE/CS3352-FDS/DGD
array([ 0.25, 0.5 , 0.75, 1. ])
 Finding index
The index is an array-like object of type [Link]
[Link]
RangeIndex(start=0, stop=4, step=1)
 Access by index
Like with a NumPy array, data can be accessed by the associated index via the familiarPython square-
bracket notation
data[1] 0.5
data[1:3]
1 0.50
2 0.75
dtype: float64
Series as generalized NumPy array
the NumPy array has an implicitly defined integer index used to access the values, the Pandas Series
has anexplicitly defined index associated with the values.
This explicit index definition gives the Series object additional capabilities. For example, the index need
not bean integer, but can consist of values of any desired type.
For example, if we wish, we can use strings as an index.
Strings as an index
data = [Link]([0.25, 0.5, 0.75, 1.0],
index=['a', 'b', 'c', 'd']) data
a 0.25
b 0.50
c 0.75
d 1.00
dtype: float64
Noncontiguous or non sequential indices.
data = [Link]([0.25, 0.5, 0.75, 1.0],
index=[2, 5, 3, 7]) data
2 0.25
5 0.50
3 0.75
7 1.00
dtype: float64
Series as specialized dictionary
A dictionary is a structure that maps arbitrary keys to a set of arbitrary values, and a Series is a structure
that maps typed keys to a set of typed values.
just as the type-specific compiled code behind a NumPy array makes it more efficient than a Python list
for certain operations, the type information of a Pandas Series makes it much more efficient than Python
dictionaries for certain operations.
We can make the Series-as-dictionary analogy even more clear by constructing a Series object directly
from aPython dictionary.
For example

TEC/CSE/CS3352-FDS/DGD
sub1={‗sai‘:90,‘ram‘:85,‘kasim‘:92,‘tamil‘:89} mark=[Link](sub1)
mark
sai 90
ram 85
kasim 92
tamil 89
dtype: int64
Dictionary-style item access
Mark[‗ram‘]
85
Array-style slicing
Mark[ ‗sai‘:‘kasim‘]
sai 90
ram 85
kasim 92
Constructing Series objects
 List or NumPy array
[Link]([2, 4, 6])
02
14
26
dtype: int64
 Repeated to fill the specified index
[Link](5, index=[100, 200, 300])

100 5
200 5
300 5
dtype: int64
 Data can be a dictionary, in which index defaults to the sorted dictionary keys
[Link]({2:'a', 1:'b', 3:'c'})

1 b
2 a
3 c
dtype: object

 The index can be explicitly set if a different result is preferred

[Link]({2:'a', 1:'b', 3:'c'}, index=[3, 2])
3c
2a
dtype: object
The Pandas DataFrame Object
The fundamental structure in Pandas is the DataFrame. The DataFrame can be thought of either as a
generalization of a NumPy array, or as a specialization of a Python dictionary.

TEC/CSE/CS3352-FDS/DGD
DataFrame as a generalized NumPy array
A DataFrame is an analog of a two-dimensional array with both flexible row indices and flexible
column names. Just as you might think of a two-dimensional array as an ordered sequence of aligned one-
dimensional columns, you can think of a DataFrame as a sequence of aligned Series objects. Here, by
―aligned‖ we mean that they share the same index.
To demonstrate this, let‘s first construct a new Series listing the marks of subject2.

sub2={'sai':91,'ram':95,'kasim':89,'tamil':90}

We can use a dictionary to construct a single two-dimensional object containing this information.
result=[Link]({'DS':sub1,'FDS':sub2})result

DS FDS
sai 90 91
ram 85 95
kasim 92 89
tamil 89 90

DataFrame has an index attribute

Like the Series object, the DataFrame has an index attribute that gives access to the index labels
[Link]
Index(['sai', 'ram', 'kasim', 'tamil'], dtype='object')
DataFrame has a columns attribute.
The DataFrame has a columns attribute, which is an Index object holding the column labels.
[Link]

Index(['DS', 'FDS'], dtype='object')

DataFrame as specialized dictionary

We can also think of a DataFrame as a specialization of a dictionary. Where a dictionary maps a key to a
value,a DataFrame maps a column name to a Series of column data.
result['DS']
sai 90
ram 85
kasim 92
tamil 89
Name:
DS,
dtype:
int64

TEC/CSE/CS3352-FDS/DGD
In a two-dimensional NumPy array, data[0] will return the first row. For a DataFrame, data['col0']
will return the first column. Because of this, it is probably better to think about DataFrames as
generalized dictionaries rather than generalized arrays, though both ways of looking at the situation can
be useful.
Constructing DataFrame objects
A Pandas DataFrame can be constructed in a variety of ways. Here we‘ll give several examples.
 From a single Series object.
 From a list of dicts.
 From a dictionary of Series objects.
 From a two-dimensional NumPy array.
 From a NumPy structured array.
From a single Series object.
A DataFrame is a collection of Series objects, and a single column DataFrame can be constructed from
a singleSeries.
sub1=[Link]({'sai':90,'ram':85,'kasim':92,'tamil':89}) [Link](sub1,columns=['DS'])
DS
sai 90
ram 85
kasim 92
tamil 89
From a list of dicts.
Any list of dictionaries can be made into a DataFrame. We‘ll use a simple list comprehension to
create somedata
data = [{'a': i, 'b': 2 * i} for i in range(3)] [Link](data)
ab000
112
224
Even if some keys in the dictionary are missing, Pandas will fill
them in with NaN (i.e.,―not a number‖) values.
[Link]([{'a': 1, 'b': 2}, {'b': 3, 'c': 4}]) a b c
0 1.0 2 NaN
1 NaN 3 4.0
From a dictionary of Series objects.
As we saw before, a DataFrame can be constructed from a dictionary of Series objects as well.
[Link]({'DS':sub1,'FDS':sub2})

DS FDS
sai 90 91
ram 85 95
kasim 92 89
tamil 89 90

From a two-dimensional NumPy array.

Given a two-dimensional array of data, we can create a DataFrame with any specified column and index

TEC/CSE/CS3352-FDS/DGD
[Link] omitted, an integer index will be used for each.
[Link]([Link](3, 2), columns=['food', 'water'],
index=['a', 'b', 'c'])

food water
a 0.865257 0.213169
b 0.442759 0.108267
c 0.047110 0.905718

From a NumPy structured array.

A Pandas DataFrame operates much like a structured array, and can be created directly.
A = [Link](3, dtype=[('A', 'i8'), ('B', 'f8')]) A

array([(0, 0.0), (0, 0.0), (0, 0.0)],

dtype=[('A', '<i8'), ('B', '<f8')])
[Link](A) A B
0 0 0.0
1 0 0.0
2 0 0.0

8. Briefly explain the hierarchical indexing with examples

Pandas does provide Panel and Panel4D objects that natively handle three-dimensional and four-
dimensional, a far more common pattern in practice is to make use of hierarchical indexing (also
known as multi-indexing)to incorporate multiple index levels within a single index.
In this way, higher-dimensional data can be compactly represented within the familiar one-dimensional
Series and two-dimensional DataFrame objects.
Here we‘ll explore the direct creation of MultiIndex objects; considerations around indexing, slicing,
and computing statistics across multiply indexed data; and useful routines for converting between simple
and hierarchically indexed representations of your data.
A Multiply Indexed Series
Pandas MultiIndex
Pandas provides a better way. Our tuple-based indexing is essentially a rudimentary multi-index, and the
Pandas MultiIndex type gives us the type of operations we wish to have. We can create a multi-index
from the tuples asfollows
index = [('California', 2000), ('California', 2010),
('New York', 2000), ('New York', 2010),
('Texas', 2000), ('Texas', 2010)]
populations = [33871648, 37253956,
18976457, 19378102,
20851820, 25145561]
pop = [Link](populations, index=index) pop
(California, 2000) 33871648
(California, 2010) 37253956
(New York, 2000) 18976457

TEC/CSE/CS3352-FDS/DGD
(New York, 2010) 19378102
(Texas, 2000) 20851820
#creating multi index
index = [Link].from_tuples(index) index

MultiIndex(levels=[['California','New York','Texas'],[2000, 2010]],

labels=[[0, 0, 1, 1, 2, 2], [0, 1, 0, 1, 0, 1]])
Hierarchical representation of the data
pop = [Link](index)pop
California 2000 33871648
2010 37253956
New York 2000 18976457
2010 19378102
Texas 2000 20851820
2010 25145561
dtype: int64
Here the first two columns of the Series representation show the multiple index values, while the third
columnshows the data.
Access all data with second index
pop[:, 2010]
California 37253956
New York 19378102
Texas 25145561 dtype: int64
MultiIndex as extra dimension
we could easily have stored the same data using a simple DataFrame with index and column
labels. The
unstack() method will quickly convert a multiplyindexed Series into a conventionally indexed
DataFrame.
pop_df = [Link]()pop_df
2000 2010
California 33871648 37253956
New York 18976457 19378102
Texas 20851820 25145561
The stack() method provides the opposite operation.
pop_df.stack()
California 2000 33871648
2010 37253956
New York 2000 18976457
2010 19378102
Texas 2000 20851820
2010 25145561
dtype: int64
Add a new column in multi dimensional data frame.
pop_df = [Link]({'total': pop,
'under18': [9267089, 9284094,

TEC/CSE/CS3352-FDS/DGD
4687374, 4318033,
5906301, 6879014]})
pop_df

total under18
California
2000 33871648 9267089
2010 37253956 9284094
New York 2000 18976457 4687374
2010 19378102 4318033
Texas 2000 20851820 5906301
2010 25145561 6879014

TEC/CSE/CS3352-FDS/DGD
 CS3352 - FOUNDATION UNITOF– I DATA SCIENCE
Question Bank
INTRODUCTION
UNIT V - DATA VISUALIZATION
Importing Matplotlib – Line plots – Scatter plots – visualizing errors – density and contour
plots – Histograms – legends – colors – subplots – text and annotation – customization –
three dimensional plotting - Geographic Data with Basemap - Visualization with Seaborn.
PART A
1. What is the purpose of matplotlib?
Matplotlib is a cross-platform, data visualization and graphical plotting library for Python and
itsnumerical extension NumPy.
One of Matplotlib’s most important features is its ability to play well with many operating
systemsand graphics backends.
2. Write the dual interface of matplotlib?
The dual interfaces of matplotlib are: a convenient MATLAB-style state-based interface, and a
morepowerful object-oriented interface.
3. How to draw a simple line plot using
matplotlib?import [Link] as plt
[Link]('seaborn-whitegrid')
import numpy
as npfig =
[Link]()
ax = [Link]()
x = [Link](0, 10,
1000)[Link](x,
[Link](x))
4. What functions can be used to draw the
scatterplot? [Link] are the functions used to
draw the scatter [Link]:
x = [Link](0, 10,
30)y = [Link](x)
[Link](x, y, 'o',
color='black');Output:

A second, more powerful method of creating scatter plots is the [Link] function, which can
beused very similarly to the [Link] function.
Example:
[Link](x, y, marker='o')
5. Write the difference between plot and scatter functions?
The primary difference of [Link] from [Link] is that it can be used to create scatter plots
where the properties of each individual point (size, face color, edge color, etc.) can be
individually controlled or mapped to data.

TEC/CSE/CS3352-FDS/DGD
6. Define contour plot?
Contour plot used to plot three dimensional data into two dimensional data. A contour plot can be
created with the [Link] function. It takes three arguments: a grid of x values, a grid of y
values, and a grid of z values.
7. What are the functions can be used to draw the contour plots?
[Link], [Link], and [Link] are the functions used to draw the contour plots.
Example:
x = [Link](0, 5, 50)
y = [Link](0, 5,
40) X, Y =
[Link](x, y)Z =
f(X, Y)
[Link](X, Y, Z,
colors='black')Output:

8. What is the purpose of using histogram?

A simple histogram is very useful in understanding a dataset. It is used to specify the
frequencydistributions between two varibales.
Example:
[Link]('seaborn-
white')
data =
[Link](1000)
[Link](data)
9. Write the source code to draw a simple
histogram?import numpy as np
import [Link]
as [Link]('seaborn-
white') data =
[Link](1000)
[Link](data)
Output:

10. How to create a three-dimensional wireframe plot?

The three-dimensional plots that work on gridded data are wireframes. It take a grid of values and

TEC/CSE/CS3352-FDS/DGD
project it onto the specified three dimensional surface, and can make the resulting three-
dimensional forms quite easy to visualize.
Example:
fig = [Link]()
ax = [Link](projection='3d')
ax.plot_wireframe(X, Y, Z,
color='black')ax.set_title('wireframe')
Output:

11. Define surface plot?

A surface plot is like a wireframe plot, but each face of the wireframe is a filled polygon. Adding a
colormap to the filled polygons can aid perception of the topology of the surface being visualized.
Example:
ax = [Link](projection='3d')
ax.plot_surface(X, Y, Z, rstride=1,
cstride=1,cmap='viridis',
edgecolor='none') ax.set_title('surface')
Output:

12. What is the use of seaborn?

Seaborn is a open-source Python library built on top of matplotlib. It is used for data visualization
and exploratory data analysis. Seaborn works easily with dataframes and the Pandas library. The
graphs created can also be customized easily.
Below are a few benefits of Data Visualization:
Graphs can help us find data trends that are useful in any machine learning or forecasting
[Link] make it easier to explain your data to non-technical people.
Visually attractive graphs can make presentations and reports much more appealing to the reader.
13. State how the labling of plots are done.
The labeling of plots includes titles, axis
labels, and simple legends.
Title - [Link]()
Label - [Link]()
[Link]()

TEC/CSE/CS3352-FDS/DGD
 Legend - [Link]()
14. How to create multiple legends?
It is only possible to create a single legend for the entire plot. If you try to create a second legend
using [Link]() or [Link](), it will simply override the first one. We can work around this by
creating a new legend artist from scratch, and then using the lower-level ax.add_artist() method to
manually add the second artist to the plot.
15. What is Color Bar?
In Matplotlib, a color bar is a separate axes that can provide a key for the meaning of colors
in a plot. Forcontinuous labels based on the color of points, lines, or regions, a labeled color bar can
be a great tool.
The simplest colorbar can be created with the [Link]() function.
16. How to create Discrete colorbars?
Colormaps are by default continuous, but
sometimes you’d like to represent discrete values.
The easiest way to do this is to use the
[Link].get_cmap() function, and pass the name of
a suitable colormap along with the number of
desired bins.
[Link](I,
cmap=[Link].get_cmap('B
lues', 6))[Link]()
[Link](-1, 1);
17. What is Subplot?
Matplotlib has the concept of subplots: groups of smaller axes that can exist together within a
single figure.
These subplots might be insets, grids of plots, or other more complicated layouts.
18. State the routines to create subplots.
There are four routines for creating subplots in Matplotlib.
 [Link]: Subplots by Hand
 [Link]: Simple Grids of Subplots
 [Link]: The Whole Grid in One Go
 [Link]: More Complicated Arrangements
19. State the different types of map projections.
The Basemap package implements several dozen such projections, all referenced by a short
format code. Here we’llbriefly demonstrate some of the more common ones.
 Cylindrical projections
 Pseudo-cylindrical projections
 Perspective projections
 Conic projections
20. Explain about Factor plots
Factor plots can be useful for this kind of
visualization as well. This allows you to
view the distribution of a parameter within
bins defined by any other parameter.

TEC/CSE/CS3352-FDS/DGD
 PART B
1. Briefly explain about the line plot
The simplest of all plots is the visualization of a single function y = f x . Here we will take a
first look at creating asimple plot of this type.
The figure (an instance of the class [Link]) can be thought of as a single container that
contains all the objectsrepresenting axes, graphics, text, and labels.
The axes (an instance of the class [Link]) is what we see above: a bounding box with ticks
and labels, which willeventually contain the plot elements that make up our visualization.
Line Colors and Styles
 The first adjustment you might wish to make to a plot is to control the line colors and
styles.
 To adjust the color, you can use the color keyword, which accepts a string argument
representing virtuallyany imaginable color. The color can be specified in a variety of
ways
 If no color is specified, Matplotlib will automatically cycle through a set of default colors
for multiple lines
Different forms of color representation.
specify color by name - color='blue'
short color code (rgbcmyk) - color='g'
Grayscale between 0 and 1 - color='0.75'
Hex code (RRGGBB from 00 to
FF) - color='#FFDD44' RGB tuple,
values 0 and 1 - color=(1.0,0.2,0.3)all HTML color names supported - col
 We can adjust the line style using the linestyle keyword.
Different line styles
linestyle='solid'
linestyle='dash
ed'
linestyle='dash
dot'
linestyle='dotte
d'
Short assignment
linestyle='-' # solid linestyle='--' # dashed linestyle='-.' # dashdot linestyle=':' # dotted

 linestyle and color codes can be combined into a single nonkeyword argument to the
[Link]() function
[Link](x, x +
0, '-g') # solid
green
[Link](x, x +
1, '--c') #
dashed cyan
[Link](x, x +
2, '-.k') #
dashdot black
[Link](x, x +
3, ':r'); # dotted

TEC/CSE/CS3352-FDS/DGD
 red

Axes Limits
 The most basic way to adjust axis limits is to use the [Link]()
and [Link]() methodsExample
[Link](1
0, 0)
[Link](1.
2, -1.2);
 The [Link]() method allows you to set the x and y limits with a single call, by
passing a list that specifies[xmin, xmax, ymin, ymax]
[Link]([-1, 11, -1.5, 1.5]);

 Aspect ratio equal is used to represent one unit in x is equal to one unit in y.
[Link]('equal')

Labeling Plots
The labeling of plots includes titles, axis
labels, and simple [Link] - [Link]()
Label -
[Link]
l()
[Link]
el()
Legend -
[Link]
()

Example programsLine color

import [Link] as plt import numpy as np
fig = [Link]() ax = [Link]()
x = [Link](0, 10, 1000) [Link](x, [Link](x));
[Link](x, [Link](x - 0), color='blue') # specify color by name [Link](x, [Link](x - 1), color='g') #
short color code (rgbcmyk) [Link](x, [Link](x - 2), color='0.75') # Grayscale between 0 and 1
[Link](x, [Link](x - 3), color='#FFDD44') # Hex code (RRGGBB from 00 to FF) [Link](x, [Link](x
- 4), color=(1.0,0.2,0.3)) # RGB tuple, values 0 and 1 [Link](x, [Link](x - 5), color='chartreuse');#
all HTML color names supported

TEC/CSE/CS3352-FDS/DGD
Line style
import [Link] as plt
import numpy as np fig = [Link]()
ax = [Link]()
x=
[Link](0,
10, 1000)
[Link](x, x +
0,
linestyle='soli
d') [Link](x,
x + 1,
linestyle='das
hed')
[Link](x, x +
2,
linestyle='das
hdot')
[Link](x, x +
3,
linestyle='dot
ted');

TEC/CSE/CS3352-FDS/DGD
# For short, you can use the
following codes:[Link](x, x + 4,
linestyle='-') # solid [Link](x, x +
5, linestyle='--') # dashed

[Link](x, x + 6, linestyle='-.') #
dashdot [Link](x, x + 7,
linestyle=':'); # dotted
Axis limit with label and legend

import
matplotl
[Link]
t as plt
import
numpy
as np
fig = [Link]() ax = [Link]()
x = [Link](0, 10, 1000)
[Link](-1, 11)
[Link](-1.5, 1.5);
[Link](x, [Link](x), '-g', label='sin(x)')
[Link](x,
[Link](x), ':b',
label='cos(x)')
[Link]("A Sine
Curve")
[Link]("x") [Link]("sin(x)"); [Link]();

2. Briefly explain about the scatter plot.

Another commonly used plot type is the simple scatter plot, a close cousin of the line plot.
Instead of points beingjoined by line segments, here the points are represented individually with
a dot, circle, or other shape.
Syntax
[Link](x, y, 'type of symbol ', color);

TEC/CSE/CS3352-FDS/DGD
 Example
[Link](x, y, 'o', color='black');
 The third argument in the function call is a character that represents the type of
symbol used for the plotting. Just as you can specify options such as '-' and '--' to
control the line style, the marker style has its own set of short string codes.
Example
 Various symbols used to specify ['o', '.', ',', 'x', '+', 'v', '^', '<', '>', 's', 'd']

 Short hand assignment of line, symbol and color also allowed.

[Link](x, y, '-ok');
 Additional arguments in [Link]()
We can specify some other parameters related with scatter plot which makes it more
attractive. They arecolor, marker size, linewidth, marker face color, marker edge color,
marker edge width, etc
Example
[Link](x, y, '-p', color='gray', markersize=15, linewidth=4, markerfacecolor='white',
markeredgecolor='gray', markeredgewidth=2) [Link](-1.2, 1.2);

Scatter Plots with [Link]

 A second, more powerful method of creating scatter plots is the [Link] function,
which can be used very similarly to the [Link] function
[Link](x, y, marker='o');
 The primary difference of [Link] from [Link] is that it can be used to create
scatter plots where the properties of each individual point (size, face color, edge color,
etc.) can be individually controlled or mapped to data.
 Notice that the color argument is automatically mapped to a color scale (shown here
by the colorbar() command), and the size argument is given in pixels.
 Cmap – color map used in scatter plot gives different color combinations.

Perceptually Uniform Sequential

['viridis', 'plasma', 'inferno', 'magma']
Sequential
['Greys', 'Purples', 'Blues', 'Greens', 'Oranges', 'Reds', 'YlOrBr',
'YlOrRd',
'OrRd', 'PuRd', 'RdPu', 'BuPu', 'GnBu', 'PuBu', 'YlGnBu', 'PuBuGn',
'BuGn','YlGn']
Sequential (2)
['binary', 'gist_yarg', 'gist_gray', 'gray', 'bone', 'pink', 'spring',
'summer',
Diverging
['PiYG', 'PRGn', 'BrBG', 'PuOr', 'RdGy', 'RdBu', 'RdYlBu', 'RdYlGn',
'Spectral','coolwarm', 'bwr', 'seismic']
Qualitative
['Pastel1', 'Pastel2', 'Paired', 'Accent', 'Dark2', 'Set1', 'Set2',
'Set3','tab10', 'tab20', 'tab20b', 'tab20c']
Miscellaneous
['flag', 'prism', 'ocean', 'gist_earth', 'terrain', 'gist_stern',
'gnuplot',
'gnuplot2', 'CMRmap', 'cubehelix', 'brg', 'hsv', 'gist_rainbow',
'rainbow','jet', 'nipy_spectral', 'gist_ncar']

TEC/CSE/CS3352-FDS/DGD
 Example programs.

Simple scatter plot.

import numpy as np
import
matplotl
[Link]
t as pltx
=
[Link]
ace(0,
10, 30)
y = [Link](x)
[Link](x, y, 'o', color='black');

Scatter plot with edge

color, face color, size,
and width of marker.
(Scatter plot with line)

import numpy as np
import [Link] as plt x = [Link](0, 10, 20)
y = [Link](x)
[Link](x, y, '-o', color='gray', markersize=15,
linewidth=4, markerfacecolor='yellow',
markeredgecolor='red', markeredgewidth=4)
[Link](-1.5, 1.5);

Scatter plot with random colors, size and transparency

import numpy as np
import [Link] as plt rng =
[Link](0)x = [Link](100)
y = [Link](100) colors = [Link](100)
sizes = 1000 * [Link](100)
[Link](x, y, c=colors, s=sizes,
alpha=0.3, map='viridis')
[Link]()

TEC/CSE/CS3352-FDS/DGD
3. Explain contour plot and histogram.
To display three-dimensional data in two dimensions using contours
or color-coded [Link] are three Matplotlib functions that can
be helpful for this task:
 [Link] for contour plots,
 [Link] for filled contour plots, and
 [Link] for showing images.

Visualizing a Three-Dimensional Function

A contour plot can be created with the [Link] function.
 a grid of x values,
 a grid of y values, and
 a grid of z values.
The x and y values represent positions on the plot, and the
The way to prepare such data is to
use the [Link] function, which
builds two-dimensional grids from
one-dimensional arrays:
Example
def f(x, y):
return [Link](x) **
10 + [Link](10 + y * x) *
[Link](x) x = [Link](0, 5,
50)
y = [Link](0, 5, 40)
X, Y = [Link](x, y)
Z = f(X, Y)
[Link](X, Y, Z, colors='black');

 Notice that by default when a single color is used, negative values are
represented by dashed lines, andpositive values by solid lines.
 Alternatively, you can color-code the lines by specifying a colormap with the cmap
argument.
 We’ll also specify that we want more lines to be drawn—20 equally spaced intervals
within the data range.
[Link](X, Y, Z, 20, cmap='RdGy');
 One potential issue with this plot is that it is a bit ―splotchy.‖ That is, the color steps
are discrete rather thancontinuous, which is not always what is desired.
 You could remedy this by setting the number of contours to a very high number, but
this results in a ratherinefficient plot: Matplotlib must render a new polygon for each
step in the level.
 A better way to handle this is to use the [Link]() function, which interprets a
two-dimensional grid ofdata as an image.

There are a few potential gotchas with imshow().

 [Link]() doesn’t accept an x and y grid, so you must manually specify the extent
[xmin, xmax, ymin, ymax] of the image on the plot.

TEC/CSE/CS3352-FDS/DGD
  [Link]() by default follows the standard image array definition where the origin
is in the upper left, notin the lower left as in most contour plots. This must be changed
when showing gridded data.
 [Link]() will automatically adjust the axis aspect ratio to match the input data;
you can change this bysetting, for example, [Link](aspect='image') to make x and y
units match.

Finally, it can sometimes be

useful to combine contour plots and
image plots. we’ll use a partially
transparent background image (with
transparency set via the alpha
parameter) and over-plot contours
with labels on the contours
themselves (using the [Link]()
function):
contours = [Link](X,
Y, Z, 3, colors='black')
[Link](contours,
inline=True, fontsize=8)
[Link](Z, extent=[0,
5, 0, 5], origin='lower',
cmap='RdGy', alpha=0.5)
[Link]();

Example Program
import numpy as np
import
matplotli
[Link]
as plt def
f(x, y):
return [Link](x) ** 10 +
[Link](10 + y * x) *
[Link](x)
x = [Link](0, 5, 50)
y = [Link](0, 5, 40) X, Y = [Link](x, y) Z = f(X, Y)
[Link](Z, extent=[0, 10, 0, 10],
origin='lower',
cmap='RdGy') [Link]()
 Histogram is the simple plot to represent the large data set. A histogram is a
graph showing frequencydistributions. It is a graph showing the number of
observations within each given interval.

Parameters
 [Link]( ) is used to plot histogram. The hist() function will use an array of numbers
to create a histogram,the array is sent into the function as an argument.

TEC/CSE/CS3352-FDS/DGD
  bins - A histogram displays numerical data by grouping data into "bins" of equal
width. Each bin is plotted as a bar whose height corresponds to how many data points
are in that bin. Bins are also sometimes called "intervals", "classes", or "buckets".
 normed - Histogram normalization is a technique to distribute the frequencies of the
histogram over a wider range than the current range.
 x - (n,) array or sequence of (n,) arrays Input values, this takes either a single array or
a sequence of arrays which are not required to be of the same length.
 histtype - {'bar', 'barstacked', 'step', 'stepfilled'}, optional The type of histogram to draw.

 'bar' is a traditional bar-type histogram. If multiple data are given the bars are
arranged side by side.
 'barstacked' is a bar-type histogram where multiple data are stacked on top of each
other.
 'step' generates a lineplot that is by default unfilled.
 'stepfilled' generates a lineplot
that is by default [Link] is 'bar'
 align - {'left',
'mid', 'right'},
optional
Controls how
the histogram is
plotted.

 'left': bars are centered on the left bin edges.

 'mid': bars are centered between the bin edges.
 'right': bars are centered
on the right bin [Link] is
'mid'
 orientation - {'horizontal', 'vertical'}, optional
If 'horizontal', barh will be used for bar-type histograms and the bottom kwarg will be the
left edges.
 color - color or array_like of colors or None, optional
Color spec or sequence of color specs, one per dataset. Default (None) uses the standard
line color sequence.

Default is None
 label - str or None, optional. Default is None

Other parameter
**kwargs - Patch properties, it allows us to pass avariable number of keyword arguments to a python
function. ** denotes this type of function.

Example
import numpy as np
import [Link] as plt
[Link]('seaborn-white')

TEC/CSE/CS3352-FDS/DGD
 data = [Link](1000)
[Link](data);

The hist() function has many options to tune both the calculation and the display; here’s an
example of a morecustomized histogram.
[Link](data, bins=30, alpha=0.5,histtype='stepfilled', color='steelblue',edgecolor='none');

The [Link] docstring has more information on other customization options available. I find
this combination of histtype='stepfilled' along with some transparency alpha to be very useful
when comparing histograms of several distributions

x1 = [Link](0, 0.8, 1000)

x2 = [Link](-2, 1, 1000)
x3 = [Link](3, 2, 1000)
kwargs = dict(histtype='stepfilled', alpha=0.3, bins=40) [Link](x1, **kwargs)

[Link](x2, **kwargs) [Link](x3, **kwargs);

Two-Dimensional Histograms and Binnings

 We can create histograms in two dimensions by dividing points among two dimensional
bins.
 We would define x and y values. Here for example We’ll start by defining some
data—an x and y arraydrawn from a multivariate Gaussian distribution:
 Simple way to plot a two-dimensional histogram is to use Matplotlib’s plt.hist2d()
function
Example
mean = [0, 0]
cov = [[1, 1], [1, 2]]
x, y = [Link].multivariate_normal(mean,
cov, 1000).T plt.hist2d(x, y, bins=30,
cmap='Blues')
cb = [Link]() cb.set_label('counts in bin')

TEC/CSE/CS3352-FDS/DGD
4. What is 3D plotting? Explain it with example.

We enable three-dimensional plots by importing the mplot3d toolkit, included with the main
Matplotlib installation.
import numpy as np
import
matplotlib.p
yplot as plt
from
mpl_toolkits
import
mplot3d fig
=
[Link]()
ax = [Link](projection='3d')

With this 3D axes enabled, we

can now plot a varietyof three-
dimensional plot types.
Three-Dimensional Points and Lines
The most basic three-dimensional plot is a line or scatter plot created from sets of (x, y, z) triples.
In analogy with the more common two-dimensional plots discussed earlier, we can create these
using the ax.plot3D
and ax.scatter3D functions

import numpy as np
import
matplotlib.p
yplot as plt

TEC/CSE/CS3352-FDS/DGD
from
mpl_toolkits
import
mplot3d ax
=
[Link](proj
ection='3d')
# Data for a
three-
dimensional
linezline =
[Link](
0, 15, 1000)
xline =
[Link](zline)
yline =
[Link](zline
)
ax.plot3D(x
line, yline,
zline, 'gray')
# Data for three-
dimensional scattered
points zdata = 15 *
[Link](100
)
xdata = [Link](zdata) + 0.1 *
[Link](100) ydata =
[Link](zdata) + 0.1 *
[Link](100)

ax.scatter3D(xdata, ydata, zdata, c=zdata,

cmap='Greens'); [Link]()

Notice that by default, the scatter points have their transparency adjusted to give a sense of depth
on the page.

Three-Dimensional Contour Plots

 mplot3d contains tools to create three-dimensional relief plots using the same inputs.
 Like two-dimensional [Link] plots, ax.contour3D requires all the input data to
be in the form of two-dimensional regular grids, with the Z data evaluated at each
point.
 Here we’ll show a three-dimensional contour diagram of a three dimensional sinusoidal
function
import numpy as np
import
matplotlib.p
yplot as plt

TEC/CSE/CS3352-FDS/DGD
from
mpl_toolkits
import
mplot3d def
f(x, y):
return
[Link]([Link](x
** 2 + y ** 2)) x
= [Link](-6,
6, 30)
y = [Link](-6, 6, 30)
X, Y = [Link](x, y) Z = f(X, Y)
fig = [Link]()
ax =
[Link](projectio
n='3d')
ax.contour3D(X,
Y, Z, 50,
cmap='binary')
ax.set_xlabel('x')
ax.set_ylabel('y') ax.set_zlabel('z') [Link]()
Sometimes the default viewing angle is not optimal, in which case we can use the
view_init method to set theelevation and azimuthal angles.
ax.view_init(60, 35) fig

Wire frames and Surface Plots

 Two other types of three-dimensional plots that work on gridded data are wireframes and
surface plots.
 These take a grid of values and project it onto the specified threedimensional
surface, and can make theresulting three-dimensional forms quite easy to visualize.

import numpy as np
import
matplotlib.p
yplot as plt
from
mpl_toolkits
import
mplot3d fig
=
[Link]()
ax =
[Link](projectio
n='3d')
ax.plot_wirefram
e(X, Y, Z,
color='black')
ax.set_title('wiref
rame');

TEC/CSE/CS3352-FDS/DGD
 [Link]()

 A surface plot is like a

wireframe plot, but each face
of the wireframe is a filled
polygon.

 Adding a colormap to the filled polygons can aid perception of the topology of the surface
being visualized

import numpy as np
import [Link] as plt from mpl_toolkits import mplot3d ax = [Link](projection='3d')
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap='viridis', edgecolor='none') ax.set_title('surface')
[Link]()

Surface Triangulations
 For some applications, the evenly
sampled grids required by the
preceding routines are overly
restrictive and inconvenient.
 In these situations, the triangulation-based plots can be very useful.
import numpy as np
import [Link] as plt from mpl_toolkits import mplot3d
theta = 2 * [Link] * [Link](1000) r = 6 * [Link](1000)
x = [Link](r * [Link](theta)) y = [Link](r * [Link](theta)) z = f(x, y)
ax = [Link](projection='3d')
[Link](x, y, z, c=z, cmap='viridis', linewidth=0.5)

5. How graphical data can be projected using matplotlib? Explain with example.

 One common type of visualization

in data science is thatof
geographic data.
 Matplotlib’s main tool for this type of visualization is the Basemap toolkit, which
is one of severalMatplotlib toolkits that live under the mpl_toolkits namespace.
 Basemap is a useful tool for Python users to have in their virtual toolbelts

TEC/CSE/CS3352-FDS/DGD
  Installation of Basemap. Once you have the Basemap toolkit installed and imported,
geographic plots alsorequire the PIL package in Python 2, or the pillow package
in Python 3.
import numpy as np
import [Link] as plt
from
mpl_toolkits.basemap
import Basemap
[Link](figsize=(8,
8))
m=
Basemap(projection='ortho',
resolution=None, lat_0=50,
lon_0=-100)
[Link](scale=0.5);

 Matplotlib axes that understands

spherical coordinates andallows us
to easily over-plot data on the map

 We’ll use an etopo image (which shows topographical features both on land and under
the ocean) as themap background
Program to display particular area of the
map with latitude andlongitude lines
import numpy as np
import [Link] as plt
from
mpl_toolkits.basemap
import Basemap from
itertools import chain
fig = [Link](figsize=(8, 8))
m = Basemap(projection='lcc', resolution=None,
width=8E6, height=8E6,
lat_0=45, lon_0=-100,)
[Link](scale=0.5, alpha=0.5)
def draw_map(m, scale=0.2):
# draw a shaded-relief image
[Link](scale=scale)
# lats and longs are returned as a dictionary
lats = [Link]([Link](-90, 90, 13))
lons = [Link]([Link](-180, 180, 13)) # keys contain the plt.Line2D
instances
lat_lines = chain(*(tup[1][0] for tup in [Link]())) lon_lines = chain(*(tup[1][0] for tup
in [Link]())) all_lines = chain(lat_lines, lon_lines)
# cycle through these lines and set the desired style for line in all_lines:
[Link](linestyle='-', alpha=0.3, color='r')

Map Projections
The Basemap package implements several dozen such projections, all referenced by a short
format code. Here we’llbriefly demonstrate some of the more common ones.

TEC/CSE/CS3352-FDS/DGD
  Cylindrical projections
 Pseudo-cylindrical projections
 Perspective projections
 Conic projections

Cylindrical projection
 The simplest of map projections are cylindrical projections, in which lines of constant
latitude and longitudeare mapped to horizontal and vertical lines, respectively.
 This type of mapping represents equatorial regions quite well, but results in
extreme distortions near thepoles.
 The spacing of latitude lines varies between different cylindrical projections,
leading to differentconservation properties, and different distortion near the poles.
 Other cylindrical projections are the Mercator (projection='merc') and the cylindrical
equal-area(projection='cea') projections.
 The additional arguments to Basemap for this view specify the latitude (lat) and
longitude (lon) of thelower-left corner (llcrnr) and upper-right corner (urcrnr) for the
desired map, in units of degrees.
import numpy as np
import [Link] as plt
from mpl_toolkits.basemap import Basemap
fig = [Link](figsize=(8, 6), edgecolor='w')
m = Basemap(projection='cyl', resolution=None, llcrnrlat=-90, urcrnrlat=90,
llcrnrlon=-180, urcrnrlon=180, ) draw_map(m)

Pseudo-cylindrical projections
 Pseudo-cylindrical projections relax the requirement that meridians (lines of
constant longitude) remainvertical; this can give better properties near the poles of
the projection.
 The Mollweide projection (projection='moll') is one common example of this, in
which all meridians areelliptical arcs
It is constructed so as to preserve area across the map: though there aredistortions near the
poles, the area of small patches reflects the true area.
Other pseudo-cylindrical projections are thesinusoidal (projection='sinu') and Robinson
(projection='robin') projections.
The extra arguments to Basemap here refer to the central latitude (lat_0) and longitude

TEC/CSE/CS3352-FDS/DGD
 (lon_0) for the desired map.
import numpy as np
import [Link] as plt
from
mpl_toolkits.basemap
import Basemap fig =
[Link](figsize=(8, 6),
edgecolor='w')
m=
Basemap(projection='moll',
resolution=None, lat_0=0,
lon_0=0)
draw_map(m)

Perspective projections
 Perspective projections are constructed using a particular choice of perspective point,
similar to if you photographed the Earth from a particular point in space (a point
which, for some projections, technically lieswithin the Earth!).
 One common example is the orthographic projection (projection='ortho'), which
shows one side of the globeas seen from a viewer at a very long distance.
 Thus, it can show only half the globe at a time.
 Other perspective-based
projections include the
gnomonic projection
(projection='gnom') and
stereographic projection
(projection='stere').
 These are often the most useful
for showing small portions of
the map.
import numpy as np
import [Link] as plt
from
mpl_toolkits.basemap
import Basemap fig =
[Link](figsize=(8,
8))
m=
Basemap(projection='ortho',
resolution=None, lat_0=50,
lon_0=0)
draw_map(m);

Conic projections
 A conic projection projects the map onto a single cone, which is then unrolled.
 This can lead to very good local properties, but regions far from the focus point of
the cone may becomevery distorted.

TEC/CSE/CS3352-FDS/DGD
  One example of this is the Lambert conformal conic projection (projection='lcc').
 It projects the map onto a cone arranged in such a way that two standard parallels
(specified in Basemap by lat_1 and lat_2) have well-represented distances, with scale
decreasing between them and increasing outsideof them.
 Other useful conic projections are the equidistant conic (projection='eqdc') and the
Albers equal-area(projection='aea') projection
import numpy as np
import [Link] as plt
from
mpl_toolkits.basemap
import Basemap fig =
[Link](figsize=(8,
8))
m = Basemap(projection='lcc', resolution=None,
lon_0=0, lat_0=50, lat_1=45, lat_2=55,
width=1.6E7, height=1.2E7) draw_map(m)

22

Drawing a Map Background

The Basemap package contains a range of useful functions for drawing borders of physical
features like continents,oceans, lakes, and rivers, as well as political boundaries such as countries
and US states and counties.
The following are some of the available drawing functions that you may wish to explore
using IPython’s helpfeatures:

 Physical boundaries and bodies of water drawcoastlines() - Draw continental coast lines
 drawlsmask() - Draw a mask between the land and sea, for use with projecting images on
one or the otherdrawmapboundary() - Draw the map boundary, including the fill color for
oceans
 drawrivers() - Draw rivers on the map
 fillcontinents() - Fill the continents with a given color; optionally fill lakes with another
color

 Political boundaries
 drawcountries() - Draw country boundaries drawstates() - Draw US state boundaries

TEC/CSE/CS3352-FDS/DGD
 drawcounties() - Draw US county boundaries

 Map features
 drawgreatcircle() - Draw a great circle between two pointsdrawparallels() - Draw lines of
constant latitude drawmeridians() - Draw lines of constant longitude drawmapscale() -
Draw a linear scale on the map
 Whole-globe images
 bluemarble() - Project NASA’s blue marble image onto the mapshadedrelief() - Project a
shaded relief image onto the map etopo() - Draw an etopo relief image onto the map
 warpimage() - Project a user-provided image onto the map

Plotting Data on Maps

 The Basemap toolkit is the ability to over-plot a variety of data onto a map background.
There are many map-specific functions available as methods of the Basemap [Link] of these
map-specific methods are:
contour()/contourf() - Draw contour lines or filled contoursimshow() - Draw an image
pcolor()/pcolormesh() - Draw a pseudocolor plot for irregular/regular meshesplot() - Draw lines
and/or markers
scatter() - Draw points with markersquiver() - Draw vectors
barbs() - Draw wind barbs drawgreatcircle() - Draw a great circle

6. State how visualization with seaborn are achieved.

The main idea of Seaborn is that it provides high-level commands to create a variety of
plot types useful forstatistical data exploration, and even some statistical model fitting.

Histograms, KDE, and densities

 In statistical data visualization, all
you want is to plot histograms and
joint distributions of variables. We
have seen that this is relatively
straightforward in Matplotlib
 Rather than a histogram, we can get a smooth
estimate of the distribution using a kernel
density estimation, which Seaborn does
with [Link]
 import pandas as pd import seaborn as sns
 data = [Link].multivariate_normal([0, 0], [[5, 2], [2,
 2]], size=2000)
 data = [Link](data, columns=['x', 'y'])
 for col in 'xy':
 [Link](data[col], shade=True)

 Histograms and KDE can be combined using distplot

 [Link](data['x'])
 [Link](data['y']);

 If we pass the full two-dimensional dataset to kdeplot, we will get atwo-dimensional

visualization of the data.
 We can see the joint distribution and the marginal distributions together using

TEC/CSE/CS3352-FDS/DGD
 [Link].

Pair plots
When you generalize joint plots to datasets of larger dimensions, you end up with pair plots.
This is very useful forexploring correlations between multidimensional data, when you’d like to
plot all pairs of values against each other.

We’ll demo this with the Iris dataset, which lists measurements of petals and sepals of three iris
species:
import seaborn as sns
iris = sns.load_dataset("iris") [Link](iris, hue='species', size=2.5);

Faceted histograms
 Sometimes the best way to view data is via histograms of subsets. Seaborn’s
FacetGrid makes thisextremely simple.
 We’ll take a look at some data that shows the amount that restaurant staff receive in
tips based on variousindicator data

TEC/CSE/CS3352-FDS/DGD
Factor plots

Factor plots can be useful for this kind of visualization as well. This allows you to view
the distribution of aparameter within bins defined by any other parameter.
Joint distributions
Similar to the pair plot we saw earlier, we can use [Link] to show the joint distribution

between differentdatasets, along with the associated marginal distributions.

Bar plots
Time series can be plotted with [Link].

TEC/CSE/CS3352-FDS/DGD