0% found this document useful (0 votes)
17 views21 pages

Numpy, Pandas, SQL: Data Analysis Guide

20 Programs and SQL Queries

Uploaded by

ps2128128
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
17 views21 pages

Numpy, Pandas, SQL: Data Analysis Guide

20 Programs and SQL Queries

Uploaded by

ps2128128
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

TABLE OF CONTENTS

[Link]. TOPIC PAGE NO.

1 Numpy and Pandas 1-5


Series
2 Pandas Dataframes 6-12
3 Data Visualisation 13-14
using Matplotlib
4 SQL Queries 15-19
1.) Write a program to create a 3x3 NumPy array with values ranging
from 2 to 10.

2.) Wap to enter 7 numbers and store only 2 digits, 3 digits numbers in a
list and then create a series NUMS from that list.
3.) Wap to enter name and price of 5 items and store in a dictionary where
name is key and price its value and then create a series PRICE from
that dictionary.

4.) Write a program to store name and percentage of 7 students in series,


where name is index label and percentage will be data values and then
do the following:
a) calculate the average percentage.
b) Store all indexes of series in P.
c) Store all values in Q
d) Display data type
e) Display shape
f) Display number of bytes of series
g) Display the last 4 values
h) Display the sum of first 3 values
i) Display element in descending order of percentage
j) Display the average of smallest 3 values
5.) Write a program to create a series object using an ndarray that has
elements in the range 40 to 60.

6.) Create series for the following:


a. to store the first 20 odd positive integers.
b. to store cube of first 10 natural numbers where number becomes
index label and cube become data values.
c. to store the square of every element of an numpy array
[44,55,66,[Link]] as data values, [1,2,3,4] as index and change
datatype to float32
7.) Create the following series to store class strength of 5 sections and do
the task directed:

a 45

b 43

c 38

d 36

e 22

a. to change the strength of section c to 42


b. to increase the strength of all sections by 2.
c. to change the strength to 40 of section from b to d as 40
d. to display the first 3 and last three sections' strength.
e. to increase strength by 2 of all sections
f. to display section names where strength is below 30.
g. To delete the information of the section permanently using del &
drop() separately.
h. To add information of new section ‘R’ with strength 40
8.) Write a Pandas program to create the following series A and B.

Series:A

Amit 70
Bob 75
Sita 60
Amaan 65

Series:B

Amit 80
Bob 50
Ratna nan
Amaan 85

Give output of following:

a) A+B b)len(A*B) c)A[::-1]+B[0:2]


d)[Link](2)+[Link](2) e)[Link](ascending=False)
9.) Following are details of 4 students' marks.

English Biology IP
Amit 79 89 87
Bob 77 78 69
Sita 76 67 77
Amaan 67 88 87

Now do the following:

a) Create dataframe marks using a 2-D dictionary where values is a list.


b) Create dataframe marks using a 2-D dictionary where values is a series.
c) Create dataframe marks using a 2-D dictionary where values is a dictionary.
d) Create dataframe marks using a list of dictionaries.
e) Create dataframe marks using nested lists.
10.) Create the Dataframe ‘ar’ as per following:
a. Data as 2-d Numpy array containing 1 to 12 integers in 3 X4 size.
b. Index names as ‘A’,’B’,’C’
c. After creation change the column names to ‘c1’,’c2’,’c3’,’c4’

11.) For the above data frame ‘ar ‘do the following :
a. Display the shape and dimension.

b. Display the index and column names.

c. Store transpose in DF.

d. To count the non-NAN values for each column and each rows
separately
e. To display number of rows

12.) For the above data frame ‘ar’ do the following :


a. To display the last 3 rows.

b. To display information from last row to first row.

c. To rename the 0 index to ‘a’ and column 1 to ‘b’.

d. To display the sum of values in the last column.


13.) Exchange first column with second [Link] a dataframe ITEMS
to store following information:

PRICE QUANTITY BRAND

SOAP 35 120 DETTOL

SANITIZER 50 [Link] SAVLON

MASK 40 1200 WILDCRAFT

NOW write statement to carry out following task:

a. to display item price and brand name using loc[ ],iloc[]


b. to display row number 2 and 3
c. to display information of mask
d. to set the quantity of sanitizer to 1500 using loc[],at[]
14.) For the above data frame ITEMS do the following:

a. to add a column AMOUNT [AMOUNT=PRICE * QUANTITY]

b. to display names of items having price between 30 and 40

c. to add anew column : {‘CODE’:[111,222,333]}

d. to delete the column CODE


15.) Create a dataframe TEST for following:

English Physics Chemistry Biology IP

Amit 79 89 88 89 87

Bob 77 56 NAN 78 69

Sita 76 NAN 38 67 77

Do the following:

a. Display the average in chemistry.


b. Display number of students present for each different subject TEST
c. Display the minimum marks in English and IP
d. Display the marks of Amit and Sita in Physics and IP.
16.) Do the following above dataframe TEST:

a. Display the names in descending order of IP marks

b. Increase the marks of all subjects by 5

c. Delete the last row.

d. Empty the Dataframe TEST.


17.) Carefully observe the following code answer as directed:
18.) The table shows passenger car fuel rates in miles per gallon for several
years. Make a LINE GRAPH of the data. Label both the axes and save the
graph with the name [Link].

YEAR: 2000 2002 2004 2006

RATE: 21.0 20.7 21.2 21.6

19.) The number of bed-sheets manufactured by a factory during five


consecutive weeks is given below.

Week First Second Third Fourth Fifth


Number of 600 850 700 300 900
Bed-sheets
Draw the bar graph representing the above data. Add suitable legends and
axes labels and title of chart.
20.) Draw histogram programmatically in python with following frequency
of scores 1,1,2,2,2,2,2,3,3,3,3,4,4,5 with range (bins) starting from 1 to 5.
1. Lakshya, an amateur database programmer is assigned a task to create a
database for a medical [Link] him by writing SQL queries for the
following questions based on the given table.
TABLE:DOCTOR
COD DNAME DEGREE SALARY DOB
E
11 AJIT KUMAR MBBS 12550 1999-12-13
22 REEMA LAGU MD 24500 2000-10-12
33 RANA SINGH MBBS 14500 2010-01-14
44 RITA LAL MD 15000 NULL
55 DINESH DM 25000 NULL
SINGH
i. Write a query to increase the salary by 20% where DOB is not known and
doctor names have at least 10 characters.
ii. Replace NULL with current date where name ends with ‘LAL’
iii. Display the information where names contain the substring ‘SINGH’.
iv. Display the third character of each name.
v. Count the doctors born in the month of October.
vi. Write a query to average salary the doctors having Md or DM degree,
vii. Write a query to display name of doctors in descending order of salary
where the date of birth is in the month of june or july.
2. Study the following table TEACHER and SALARY and write the
MySQL commands for the questions (i) and (ii) and give outputs for
MySQL queries (iii) and (iv).

Table : TEACHER
SID NAME DEPT gender EXPERIENCE

101 Siddharth Computer M 12

104 Raghav Physics M 5


107 Naman Chemistry M 10

114 Nupur Computer F 3

109 Janvi Physics F 9

105 Rama Accounts M NULL

117 James Computer F 3

111 Binoy Accounts F 12

130 Samuel NULL M NULL

Table: Salary
SID BASIC ALLOWANCE DA
101 12000 1000 300
104 23000 2300 500
107 32000 4000 500
114 12000 5200 1000
109 42000 1700 200
105 18900 1690 300
130 21700 2600 300

(i) Insert a record 102,”mohan”,MATHS”,’M’, 15 in table TEACHER.


(ii) Display all the records in decreasing order of experience.

(iii) Display all the records in from salary where DA is between 100 and 500.
(iv) Increase basic salary by 20% where experience is more than 10.

(v) Delete the records name contains letter ‘o’ from teacher table.

(vi) Count the number of unique departments in teacher table

(vii) Display the number of male and female teachers separately from teacher
table.

(viii) Display NAME and DA of all staff who are in Accounts department and
having more than 10 years of experience and DA is more than 300.

(ix) Display the NAME and SALARY of all staff whose salary is more than average
salary. (SALARY=BASIC +ALLOWANCE+ DA)

(x) Display the minimum and maximum allowance from salary table.

(xi)Display the average basic salary of female teachers.


(xiii) Increase the DA by 20% of female teachers.
Give outputs:

(xiv) select left(name,3),right(dept,2) from teacher;

(xv) Select avg(experience) from teacher;

(xvi) Select gender,count(*) from teacher group by gender;

(xvii) Select name,allowance*0.20 from teacher ,salary where [Link]


=[Link] and dept in(‘’CHEMISTRY’,’physics’);

(Xix)Select count(distinct dept) from teacher A,salary B where [Link]=[Link];

Common questions

Powered by AI

In pandas, missing data is managed using NaN within DataFrame and Series objects, with built-in functions like fillna or dropna designed for dynamic data cleaning . This allows easy imputation or exclusion during analytical processes. In contrast, SQL databases handle nulls using specific queries and conditions, such as IS NULL, which require explicit conditional management during data retrieval and manipulation . This demands a greater focus on query design in SQL to avoid errors in reports. Pandas provides more straightforward, flexible interfaces for user-driven data exploration and handling missing data compared to SQL's set-based operations .

Creating pandas DataFrames using lists of dictionaries offers flexibility in defining heterogeneous data structures and is easily scalable by adding more dictionary entries . This method is particularly suited for larger datasets due to its ease of conversion from JSON-like data. In contrast, using 2-D dictionaries is more intuitive for aligning related data without additional processing but can become complex in maintenance as datasets grow due to potential hierarchical depth. Using numpy arrays for DataFrame construction is highly efficient for homogeneous numerical data due to performance benefits, although less flexible when dealing with non-numerical data types .

Dynamic salary adjustments can be efficiently managed by integrating conditional SQL statements that evaluate personal details. For instance, increasing the salary by 20% for doctors where DOB is unknown involves a query: 'UPDATE DOCTOR SET SALARY = SALARY * 1.2 WHERE DOB IS NULL;'. This ensures only targeted entries are modified . Applying checks on string length in names enhances accuracy, using 'LENGTH(DNAME) >= 10'. Employing conditional updates facilitates precise management of adjustments without broad application, crucial for maintaining database integrity in sensitive environments like healthcare .

Numpy arrays are more efficient in terms of performance and memory usage compared to lists when dealing with large datasets in Python. They offer vectorized operations that are significantly faster for numerical computations, which can be crucial in data science tasks such as creating dataframes or conducting data visualizations with large datasets . They also support multi-dimensional arrays, enabling complex mathematical operations without native support in lists. This enhances their utility in tasks like computing statistics or processing large matrix operations, which are common in analytical tasks discussed in the document . However, they are less flexible with mixed data types and require additional handling for complex data structures like DataFrames in pandas.

To handle null values in SQL efficiently, conditional statements such as IFNULL or COALESCE can be used to replace nulls with a default value or current date, especially important for fields like DOB to infer missing data . When handling date-time data, ensuring the consistency of date formats and using functions like STR_TO_DATE can prevent errors. Analyzing queries such as 'Replace NULL with current date where name ends with ‘LAL’' highlights the need for precise handling to avoid logical errors and data integrity issues . Potential pitfalls include improper handling of null values leading to erroneous reporting and difficulties in scheduling operations dependent on date fields.

Using a Series as values in a DataFrame allows for labeled, one-dimensional data which provides index labels. This can facilitate operations such as aligning on indices, whereas using lists requires explicit management of structure and indices. A Series inherently handles missing data and supports operations like arithmetic operations with index labels . When using a 2-D dictionary, the flexibility is greater because it allows for hierarchical data structures. In contrast, lists are more rigid and require additional handling for operations that require index alignment .

Bar graphs effectively represent categorical data with discrete time intervals, visualizing trends over specific periods rather than continuous changes. For manufacturing settings, weeks can be represented as separate bars to clearly show the number of bed-sheets produced, emphasizing quantity comparison across different periods . However, line graphs better depict trends and changes over time when precision in continuity is needed. Bar graphs can obscure subtle fluctuations in data, making them less suitable for detailed time-series analysis without additional enhancement like annotations or granularity adjustments .

Matplotlib offers extensive customization and precise control over plot elements, making it highly effective for complex dataset visualization like manufacturing outputs with varied metrics . It supports a wide range of plots, from basic line graphs to annotation-rich bar and pie charts, essential for illustrating production variances across weeks . However, alternatives like Seaborn provide more aesthetically pleasing defaults suited for statistical insights without requiring extensive customization setups. While Matplotlib requires more initial setup, its flexibility allows it to handle complex data visualization tasks that involve multiple KPIs more comprehensively than more restrictive, higher-level tools .

Pandas Series inherently accommodate NaN values and handle them gracefully during arithmetic operations by defaulting results to NaN if any operand is NaN . This allows continuing operations without intervention, an advantage over complete numerical datasets which might require pre-processing. Handling complete datasets involves additional checks or filters to prevent errors during calculations, which can increase computational overhead if not done efficiently. NaN handling is embedded within pandas functions, which can streamline workflows when consistency in calculations across datasets with missing values is needed, such as in calculating averages or sums where NaN values should not skew results .

Manually adjusting index labels and column names enhances DataFrame clarity and operational efficiency, as seen in the creation of the DataFrame 'ar' with custom indices and renamed columns . Custom indices improve data accessibility when conducting specific data manipulations, enabling more intuitive sorting and retrieval. However, manually assigning indices can lead to inconsistencies if not uniformly applied across related datasets, reducing adaptability when merging DataFrames or performing aggregate operations. Additionally, mismanagement in manual settings can cause index misalignment, leading to potential data integrity issues during operations such as concatenation or resampling .

You might also like