Scribd
Upload
38 views7 pages

Pandas Exercises with Datasets Analysis

The document provides a series of exercises using Pandas to work with and analyze various CSV datasets. It includes questions about slicing and indexing DataFrames, working with multi-index objects, transforming data between wide and long formats, and performing operations like grouping, filtering and aggregating.

Uploaded by

Durga Devi P
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
38 views7 pages

Pandas Exercises with Datasets Analysis

The document provides a series of exercises using Pandas to work with and analyze various CSV datasets. It includes questions about slicing and indexing DataFrames, working with multi-index objects, transforming data between wide and long formats, and performing operations like grouping, filtering and aggregating.

Uploaded by

Durga Devi P
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Exercises

1. Use the [Link] dataset and run the code shown in the following
screenshots. Then, answer the questions that follow:
Figure 1.48 – Exercise 1

a) Use the output to answer what is the difference in behavior


of .loc and .iloc when it comes to slicing?

b) Without running, but just by looking at the data, what will be the
output of adult_df.loc['10000':'10003', 'relationship':'sex']?

c) Without running, but just by looking at the data, what will be the
output of adult_df.iloc[0:3, 7:9]?

2. Use Pandas to read [Link] into adult_df and then use


the .groupby() function to run the following code and create the multi-
index series mlt_sr:

3. import pandas as pd
4. adult_df = pd.read_csv('[Link]')
5. mlt_seris =adult_df.groupby(['race','sex','income']).[Link]()
mlt_seris

a) Now that you have created a multi-index series, run the following
code, study the outputs, and answer the following questions:

Run the following code first and then answer this question: When we
use .iloc[] for a multi-index series or DataFrame, what should we expect?

print(mlt_seris.iloc[0])
print(mlt_seris.iloc[1])
print(mlt_seris.iloc[2])

b) Run the following code first and then answer this question: When we
use .loc[] to access the data of one of the innermost index levels of the
multi-index series, what should we expect?

mlt_seris.loc['Other']
c) Run the following code first and then answer this question: When we
use .loc[] to access the data of one of the non-innermost index levels of a
multi-index series, what should we expect?

When you run either line of the following code, you will get an error, and
that is the point of this question. Study the error and try to answer the
question:

mlt_seris.loc['Other']
mlt_seris.loc['<=50K']

d) Run the following code first and then answer this question: How does
the use of .loc[] or .iloc[] differ when working with a multi-index series
or a DataFrame?

print(mlt_seris.loc['Other']['Female']['<=50K'])
print(mlt_seris.iloc[12])

6. For this exercise, you need to use a new dataset: [Link].


Visit [Link] and see the latest song
rankings of the day. This dataset presents information and rankings for
317 song tracks in 80 columns. The first four columns
are artist, track, time, and date_e. The first columns are intuitive
descriptions of song tracks. The date_e column shows the date that the
songs entered the hot 100 list. The rest of the 76 columns are song
rankings at the end of each week from "w1" to "w76". Download and
read this dataset using Pandas and answer the following questions:

a) Write one line of code that gives you a great idea of how many null
values each column has. If any columns have no non-null values, drop
them.

b) With a for loop, draw and study the values in each of the remaining W
columns.

c) The dataset is in wide format. Use an appropriate function to switch to


a long format and name the transformed DataFrame mlt_df.
d) Write code that shows mlt_df every 1,200 rows.

e) Run the following code first and answer this question: Could this also
have been done by using Boolean masking?

mlt_df.query('artist == "Spears, Britney"')

f) Use either the approach in e or the Boolean mask to extract all the
unique songs that Britney Spears has in this dataset.

g) In mlt_df, show all of the weeks when the song "Oops!.. I Did It
Again" was in the top 100.

7. We will use [Link] for this exercise. Each row of this dataset
shows an hourly measurement recording of one of the following five air
pollutants: NO, NO2, NOX, PM10, and PM2.5. The data was collected in
a location in London for the entirety of the year 2017. Read the data using
Pandas and perform the following tasks:
a) The dataset has six columns. Three of them, named 'Site', 'Units', and
'Provisional or Ratified' are not adding any informational values as they
are the same across the whole dataset. Use the following code to drop
them:

air_df.drop(columns=['Site','Units','Provisional or Ratified'],
inplace=True)

b) The dataset is in a long format. Apply the appropriate function to


switch it to the wide format. Name the transformed Dataframe pvt_df.

c) Draw and study the histogram and boxplots for columns of pvt_df.
8. We will continue working with [Link]:
a) Run the following code, see its output, and then study the code to
answer what each line of this code does:

air_df = pd.read_csv('[Link]')
air_df.drop(columns=['Site','Units','Provisional or Ratified'],
inplace=True)
datetime_df = air_df.[Link](' ',expand=True)
datetime_df.columns = ['Date','Time']
date_df = datetime_df.[Link]('/',expand=True)
date_df.columns = ['Day','Month','Year']
air_df =
air_df.join(date_df).join(datetime_df.Time).drop(columns=['ReadingDate
Time','Year'])
air_df

b) Run the following code, see its output, and then study the code to
answer what this line of code does:

air_df = air_df.set_index(['Month','Day','Time','Species'])
air_df

c) Run the following code, see its output, and then study the code to
answer what this line of code does:

air_df.unstack()

d) Compare the output of the preceding code with pvt_df from Exercise
4. Are they the same?

e) Explain what the differences and similarities are between the


pair .melt()/.pivot() and the pair .stack()/.unstack()?

f) If you were to choose one counterpart


for .melt() between .stack()/.unstack(), which one would you choose?

Common questions

Powered by AI

In a multi-index series, using .loc on non-innermost index levels can result in ambiguities or errors if not properly specified, because it expects all remaining levels to be specified. Conversely, accessing data from the innermost index is straightforward since .loc is capable of directly retrieving the data based on the specified innermost level .

Setting an index with multiple columns in Pandas results in a multi-index or hierarchical index, which facilitates complex data analysis by allowing multi-level sorting and subsetting. This structure can lead to more organized and efficient data handling, enabling targeted operations like groupby calculations and advanced filtering strategies. Multi-index structures work well for time-series data or datasets requiring multiple categorical hierarchies .

.melt() and .pivot() are functions used primarily for reshaping data from wide to long format and vice versa. In contrast, .stack() and .unstack() deal with hierarchical indexing by reshaping the data based on the level of index or column you specify. While .melt()/.pivot() focuses on columns to rows transformations and vice versa, .stack()/.unstack() manipulates multi-index structures, essentially flipping levels into columns or rows .

When using a for loop to plot values from a DataFrame, considerations include selecting appropriate plot types for the data, ensuring axes are labeled clearly, using titles and legends to increase readability, and choosing colors that distinguish different datasets or variables. Moreover, handling missing data correctly and scaling plots to fit within visible boundaries are crucial for clarity and for providing meaningful insight .

When using .iloc[] on a multi-index series or DataFrame, the method allows access using integer-location based indexing without regards to the levels of indices. This means it will access rows based on their position within the data structure, potentially allowing you to access first-n elements or particular row positions in the overall DataFrame or series .

While both .loc and .iloc are used for slicing data in a DataFrame, they operate differently. .loc is label-based, meaning it will include the endpoints specified in the slice, whereas .iloc is integer-location based and excludes the last endpoint. For example, .loc allows access using labels and may include both endpoints in slices if labels align, while .iloc uses position index, much like list slicing in Python which does not include the final index .

To identify columns that contain entirely null values in a dataset using Pandas, one can use the .isnull() method combined with .all() to check for full columns of null values. After identifying such columns, they can be dropped using the .dropna(axis=1, how='all') method, which removes columns where all values are missing, optimizing the DataFrame for further analysis .

To extract specific artist data efficiently from a long-format DataFrame using Pandas, methods such as Boolean masking or the .query() method can be utilized. This typically involves creating a condition that filters the DataFrame where the 'artist' column matches a specific name. For instance, mlt_df.query('artist == "Spears, Britney"') would retrieve all rows corresponding to that artist, ensuring efficient selection of relevant data .

Without executing the code and assuming 'adult_df' is indexed with default behavior, 'adult_df.loc['10000':'10003', 'relationship':'sex']' would provide the rows with index labels from '10000' to '10003' (inclusive) and columns from 'relationship' to 'sex' (inclusive) because .loc uses label-based indexing .

To convert a dataset from a wide format to a long format in Pandas, you can use the .melt() function. This method ‘un-pivots’ a dataset from a wide format to a long format by collapsing columns into rows while retaining or specifying the identifier variables (id_vars). This transformation facilitates data analysis and visualization tasks as it allows flexible manipulation of observations .

You might also like