Python

Master Data Manipulation

&
Visualization

Prepared by: Syed Afroz Ali

Data Scientist (Kaggle Grandmaster)
Data Manipulation in Python for Data Analysis
Kaggle Notebook: : Prepared by: Syed Afroz Ali (Kaggle Grandmaster)

[Link]
Follow for more AI content: [Link]

import pandas as pd
import numpy as np
import seaborn as sns
import [Link] as px
import [Link] as plt
%matplotlib inline

[Link](rc={"[Link]":"Beige" , "[Link]" : False})

import warnings
[Link]("ignore")

# Set Display

pd.set_option('display.max_columns',None)
pd.set_option('display.max_rows',None)
pd.set_option('[Link]', 2)

# Check Library Version

import numpy
print('numpy:{}'.format(numpy.__version__))

numpy:1.26.4

# Load the dataset

df = pd.read_csv("[Link]")
display([Link])
[Link]()

(891, 12)
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

Cumings, Mrs. John Bradley (Florence

1 2 1 1 female 38.0 1 0 PC 17599 71.28 C85 C
Briggs Th...

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282

Futrelle, Mrs. Jacques Heath (Lily May

3 4 1 1 female 35.0 1 0 113803 53.10 C123 S
Peel)

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

# Set Table Properties

[Link](3).style.set_properties(**{'background-color': 'blue',
'color': 'white',
'border-color': 'darkblack'})

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.000000 1 0 A/5 21171 7.250000 nan S

Cumings, Mrs. John Bradley

1 2 1 1 female 38.000000 1 0 PC 17599 71.283300 C85 C
(Florence Briggs Thayer)

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.000000 0 0 7.925000 nan S
3101282

# Replacing Values/Names in a Column:

df1 = [Link]('Deep')
df1["Survived"].replace({0:"Died" , 1:"Saved"},inplace = True)
[Link](3)
 PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 Died 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

Cumings, Mrs. John Bradley (Florence

1 2 Saved 1 female 38.0 1 0 PC 17599 71.28 C85 C
Briggs Th...

STON/O2.
2 3 Saved 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282

# Drop Columns
df1 = [Link]('Deep')
df1 = [Link](['PassengerId','Ticket'],axis=1)
[Link](3)

Survived Pclass Name Sex Age SibSp Parch Fare Cabin Embarked

0 0 3 Braund, Mr. Owen Harris male 22.0 1 0 7.25 NaN S

1 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 71.28 C85 C

2 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S

# Drop Rows

df = [Link](labels=[1,3,5,7],axis=0)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

Johnson, Mrs. Oscar W (Elisabeth

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
Vilhelmina Berg)

# Missing Value check

print('Method 1:')
[Link]().sum()

Method 1:
PassengerId 0
Survived 0
Pclass 0
Name 0
Sex 0
Age 176
SibSp 0
Parch 0
Ticket 0
Fare 0
Cabin 685
Embarked 2
dtype: int64

var1 = [col for col in [Link] if df[col].isnull().sum() != 0]

print(df[var1].isnull().sum())

Age 176
Cabin 685
Embarked 2
dtype: int64

# Missing Value check

print('Method 12:')
import missingno as msno
[Link](df)
[Link]()

Method 12:
df[df['Embarked'].isnull()]

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

61 62 1 1 Icard, Miss. Amelie female 38.0 0 0 113572 80.0 B28 NaN

829 830 1 1 Stone, Mrs. George Nelson (Martha Evelyn) female 62.0 0 0 113572 80.0 B28 NaN

sample_incomplete_rows =df[[Link]().any(axis=1)].head()
sample_incomplete_rows

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

Johnson, Mrs. Oscar W (Elisabeth

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
Vilhelmina Berg)

9 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) female 14.0 1 0 237736 30.07 NaN C

[Link]()

PassengerId Survived Pclass Age SibSp Parch Fare

count 887.00 887.00 887.00 711.00 887.00 887.00 887.00

mean 447.99 0.38 2.31 29.72 0.52 0.38 32.18

std 256.22 0.49 0.83 14.52 1.10 0.81 49.78

min 1.00 0.00 1.00 0.42 0.00 0.00 0.00

25% 226.50 0.00 2.00 20.25 0.00 0.00 7.90

50% 448.00 0.00 3.00 28.00 0.00 0.00 14.45

75% 669.50 1.00 3.00 38.00 1.00 0.00 31.00

max 891.00 1.00 3.00 80.00 8.00 6.00 512.33

# Describe
df[df['Survived']==0].describe().[Link].background_gradient(subset=['mean','std','50%','count'], cmap='RdPu')
 count mean std min 25% 50% 75% max

PassengerId 547.000000 448.625229 259.750818 1.000000 213.500000 457.000000 675.500000 891.000000

Survived 547.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000

Pclass 547.000000 2.530165 0.736605 1.000000 2.000000 3.000000 3.000000 3.000000

Age 423.000000 30.693853 14.120135 1.000000 21.000000 28.000000 39.000000 74.000000

SibSp 547.000000 0.550274 1.286281 0.000000 0.000000 0.000000 1.000000 8.000000

Parch 547.000000 0.329068 0.824052 0.000000 0.000000 0.000000 0.000000 6.000000

Fare 547.000000 22.144765 31.440164 0.000000 7.854200 10.500000 26.000000 263.000000

[Link](percentiles=[0.05,0.25,0.35,0.5,0.75,0.85,0.95,0.995,0.999])

PassengerId Survived Pclass Age SibSp Parch Fare

count 887.00 887.00 887.00 711.00 887.00 887.00 887.00

mean 447.99 0.38 2.31 29.72 0.52 0.38 32.18

std 256.22 0.49 0.83 14.52 1.10 0.81 49.78

min 1.00 0.00 1.00 0.42 0.00 0.00 0.00

5% 49.30 0.00 1.00 4.00 0.00 0.00 7.22

25% 226.50 0.00 2.00 20.25 0.00 0.00 7.90

35% 315.10 0.00 2.00 24.00 0.00 0.00 9.00

50% 448.00 0.00 3.00 28.00 0.00 0.00 14.45

75% 669.50 1.00 3.00 38.00 1.00 0.00 31.00

85% 758.10 1.00 3.00 45.00 1.00 1.00 56.50

95% 846.70 1.00 3.00 56.00 2.70 2.00 112.56

99.5% 886.57 1.00 3.00 70.73 8.00 5.00 263.00

99.9% 890.11 1.00 3.00 75.74 8.00 5.11 512.33

max 891.00 1.00 3.00 80.00 8.00 6.00 512.33

# Agg
df[['Age','Fare','Pclass']].agg(['sum','max','mean','std','skew','kurt'])

Age Fare Pclass

sum 21130.17 28540.03 2049.00

max 80.00 512.33 3.00

mean 29.72 32.18 2.31

std 14.52 49.78 0.83

skew 0.40 4.79 -0.63

kurt 0.18 33.33 -1.27

# value_counts
df['Embarked'].value_counts().to_frame()

count

Embarked

S 642

C 167

Q 76

df['Embarked'].value_counts().tolist()

[642, 167, 76]

# value_counts for Multiple Columns

for col in df[['Survived','Sex','Embarked']]:
print(df[col].value_counts().to_frame())
print("****"*7)
 count
Survived
0 547
1 340
****************************
count
Sex
male 575
female 312
****************************
count
Embarked
S 642
C 167
Q 76
****************************

#Count
df[['Age','Embarked','Sex']].count()

Age 711
Embarked 885
Sex 887
dtype: int64

df['Embarked'][df['Sex']=='female'].value_counts(normalize=True)*100

Embarked
S 65.16
C 23.23
Q 11.61
Name: proportion, dtype: float64

df['Embarked'].value_counts()/len(df['Embarked'])

Embarked
S 0.72
C 0.19
Q 0.09
Name: count, dtype: float64

#Shuffling the data

df2 = [Link](frac=1,random_state=3)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

733 734 0 2 Berriman, Mr. William John male 23.0 0 0 28425 13.00 NaN S

95 96 0 3 Shorney, Mr. Charles Joseph male NaN 0 0 374910 8.05 NaN S

Watt, Mrs. James (Elizabeth "Bessie" C.A.

161 162 1 2 female 40.0 0 0 15.75 NaN S
Inglis Mi... 33595

392 393 0 3 Gustafsson, Mr. Johan Birger male 28.0 2 0 3101277 7.92 NaN S

614 615 0 3 Brocklebank, Mr. William Alfred male 35.0 0 0 364512 8.05 NaN S

df1 = [Link]()

columns = ['Age']

for col in columns:

df1[col].replace(0, [Link], inplace=True)

[Link]()

PassengerId Survived Pclass Age SibSp Parch Fare

count 887.00 887.00 887.00 711.00 887.00 887.00 887.00

mean 447.99 0.38 2.31 29.72 0.52 0.38 32.18

std 256.22 0.49 0.83 14.52 1.10 0.81 49.78

min 1.00 0.00 1.00 0.42 0.00 0.00 0.00

25% 226.50 0.00 2.00 20.25 0.00 0.00 7.90

50% 448.00 0.00 3.00 28.00 0.00 0.00 14.45

75% 669.50 1.00 3.00 38.00 1.00 0.00 31.00

max 891.00 1.00 3.00 80.00 8.00 6.00 512.33

corr = df.select_dtypes('number').corr()
display(corr)

[Link](corr, annot=True, cmap='viridis')

[Link]('Features')
[Link]('Features')
[Link]('Correlation Heatmap')
[Link]()

PassengerId Survived Pclass Age SibSp Parch Fare

PassengerId 1.00e+00 -3.15e-03 -0.04 0.03 -0.05 -2.95e-03 0.01

Survived -3.15e-03 1.00e+00 -0.33 -0.08 -0.04 8.35e-02 0.26

Pclass -3.84e-02 -3.35e-01 1.00 -0.37 0.08 1.66e-02 -0.55

Age 3.49e-02 -8.15e-02 -0.37 1.00 -0.30 -1.87e-01 0.09

SibSp -5.30e-02 -3.52e-02 0.08 -0.30 1.00 4.15e-01 0.16

Parch -2.95e-03 8.35e-02 0.02 -0.19 0.41 1.00e+00 0.22

Fare 1.38e-02 2.56e-01 -0.55 0.09 0.16 2.17e-01 1.00

corr = [Link](["Embarked"])[["Fare", "Age"]].corr()

display(corr)

n_groups = len([Link][0])

fig, axes = [Link](nrows=1, ncols=min(3, n_groups), figsize=(12, 3))

group_count = 0
for embarked_group in [Link][0]:
ax = [Link][group_count]
[Link]([Link](embarked_group), annot=True, cmap='viridis', ax=ax)
ax.set_title(f"Correlation Heatmap for Embarked: {embarked_group}")
group_count += 1

plt.tight_layout()
[Link]()

Fare Age

Embarked

C Fare 1.00 0.16

Age 0.16 1.00

Q Fare 1.00 0.03

Age 0.03 1.00

S Fare 1.00 0.05

Age 0.05 1.00

X = df.select_dtypes('number').drop(['Survived'],axis=1)
y = df.select_dtypes('number')['Survived']
[Link](y).[Link](
figsize = (16, 4), title = "Correlation with Survived", fontsize = 15,
rot = 90, grid = True)
[Link]()

corr = df.select_dtypes('number').corr()
mask = [Link](np.ones_like(corr,dtype = bool))
[Link](dpi=100)
[Link]('Correlation Analysis')
[Link](corr,mask=mask,annot=True,lw=0,linecolor='white',cmap='viridis',fmt = "0.2f")
[Link](rotation=90)
[Link](rotation = 0)
[Link]()
abs(df.select_dtypes('number').corr()).style.highlight_min(axis=0)

PassengerId Survived Pclass Age SibSp Parch Fare

PassengerId 1.000000 0.003153 0.038432 0.034866 0.052966 0.002953 0.013752

Survived 0.003153 1.000000 0.334575 0.081455 0.035186 0.083506 0.255761

Pclass 0.038432 0.334575 1.000000 0.367249 0.083521 0.016574 0.548991

Age 0.034866 0.081455 0.367249 1.000000 0.304297 0.187338 0.094965

SibSp 0.052966 0.035186 0.083521 0.304297 1.000000 0.414724 0.159978

Parch 0.002953 0.083506 0.016574 0.187338 0.414724 1.000000 0.217091

Fare 0.013752 0.255761 0.548991 0.094965 0.159978 0.217091 1.000000

df1 = df1[df1['Cabin'].notna()]
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

10 11 1 3 Sandstrom, Miss. Marguerite Rut female 4.0 1 1 PP 9549 16.70 G6 S

11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 C103 S

21 22 1 2 Beesley, Mr. Lawrence male 34.0 0 0 248698 13.00 D56 S

23 24 1 1 Sloper, Mr. William Thompson male 28.0 0 0 113788 35.50 A6 S

print('Passenger Survived in Titanic: {:d}'.format(df1['Survived'].value_counts()[0]))

print('Passenger Died in Titanic: {:d}'.format(df1['Survived'].value_counts()[1]))

Passenger Survived in Titanic: 68

Passenger Died in Titanic: 134

print('Survived sample ratio: {:.3f} %'.format(df1['Survived'].value_counts()[0]/len(df1)*100))

print('Died sample ratio: {:.3f} %'.format(df1['Survived'].value_counts()[1]/len(df1)*100))

Survived sample ratio: 33.663 %

Died sample ratio: 66.337 %

# Fillna Method

df1 = [Link]()
df1 = [Link]()

# Fillna Method

df1 = [Link]()
[Link](method="ffill", inplace=True)

# Fill Null Values by Mean Value

df1 = [Link]()
df1["Age"] = df1["Age"].fillna(df1["Age"].mean())

# Fill Null Values by Desiresd Value

df1 = [Link]()
df1['Embarked'] = df1['Embarked'].fillna(df1['Embarked'] == 'Q')
[Link](3)

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.92 NaN S

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

#Fill Method :
df1 = [Link]()
df1['Age'] = df1['Age'].fillna(0)
df1['Age'] = df1['Age'].fillna('None')
df1["Age"].fillna(method="backfill",inplace=True)
df1["Embarked"].fillna(value="A",inplace=True)
df1["Pclass"].fillna(value= 0,inplace=True)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

Johnson, Mrs. Oscar W (Elisabeth

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
Vilhelmina Berg)

# Find Method/Select Method

# Find All Null Values in the dataframe

df1 = [Link]()
df1 = [Link]('Cabin',axis =1)

sample_incomplete_rows = df1[[Link]().any(axis=1)]
display(sample_incomplete_rows.shape)
sample_incomplete_rows.head()

(178, 11)
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Embarked

17 18 1 2 Williams, Mr. Charles Eugene male NaN 0 0 244373 13.00 S

19 20 1 3 Masselmani, Mrs. Fatima female NaN 0 0 2649 7.22 C

26 27 0 3 Emir, Mr. Farred Chehab male NaN 0 0 2631 7.22 C

28 29 1 3 O'Dwyer, Miss. Ellen "Nellie" female NaN 0 0 330959 7.88 Q

29 30 0 3 Todoroff, Mr. Lalio male NaN 0 0 349216 7.90 S

titanic_Fare500 = df1[df1['Fare'] > 500][['Name','Embarked']]

display(titanic_Fare500.shape)
titanic_Fare500

(3, 2)
Name Embarked

258 Ward, Miss. Anna C

679 Cardeza, Mr. Thomas Drake Martinez C

737 Lesurer, Mr. Gustave J C

titanic_age_70 = [Link][df1['Age'] > 70, ["Name","Embarked","Sex"]]

display(titanic_age_70.shape)
titanic_age_70

(5, 3)
 Name Embarked Sex

96 Goldschmidt, Mr. George B C male

116 Connors, Mr. Patrick Q male

493 Artagaveytia, Mr. Ramon C male

630 Barkworth, Mr. Algernon Henry Wilson S male

851 Svensson, Mr. Johan S male

titanic_age_selection = df[(df["Sex"] == "male") & (df["Age"] > 50.00)]

display(titanic_age_selection.shape)
titanic_age_selection.head()

(47, 12)
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

33 34 0 2 Wheadon, Mr. Edward H male 66.0 0 0 C.A. 24579 10.50 NaN S

54 55 0 1 Ostby, Mr. Engelhart Cornelius male 65.0 0 1 113509 61.98 B30 C

94 95 0 3 Coxon, Mr. Daniel male 59.0 0 0 364500 7.25 NaN S

96 97 0 1 Goldschmidt, Mr. George B male 71.0 0 0 PC 17754 34.65 A5 C

women = [Link][df1['Sex'] == 'female']["Survived"]

rate_women = ([Link]()/len(women)).round(3)*100
print("Percentage of women who survived:",rate_women,"%")

Percentage of women who survived: 74.0 %

men = [Link][df1['Sex'] == 'male']["Survived"]

rate_men = ([Link]()/len(men)).round(3)*100
print("Percentage of Men who survived:", rate_men,"%")

Percentage of Men who survived: 19.0 %

titanic_Pclass = df1[df1["Pclass"].isin([1, 2])]

display(titanic_Pclass.shape)
titanic_Pclass.head()

(398, 11)
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Embarked

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 S

9 10 1 2 Nasser, Mrs. Nicholas (Adele Achem) female 14.0 1 0 237736 30.07 C

11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 S

15 16 1 2 Hewlett, Mrs. (Mary D Kingcome) female 55.0 0 0 248706 16.00 S

17 18 1 2 Williams, Mr. Charles Eugene male NaN 0 0 244373 13.00 S

df1 = [Link]()
cabin_no_na = df1[df1["Cabin"].notna()]
display(cabin_no_na.shape)
cabin_no_na.head()

(202, 12)
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

10 11 1 3 Sandstrom, Miss. Marguerite Rut female 4.0 1 1 PP 9549 16.70 G6 S

11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 C103 S

21 22 1 2 Beesley, Mr. Lawrence male 34.0 0 0 248698 13.00 D56 S

23 24 1 1 Sloper, Mr. William Thompson male 28.0 0 0 113788 35.50 A6 S

titanic_Pclass = df1[(df1["Pclass"] == 1) & (df1["Sex"] == 'female') & (df1["Age"] > 50 ) ]

display(titanic_Pclass.shape)
titanic_Pclass.head()

(13, 12)
 PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 C103 S

PC
195 196 1 1 Lurette, Miss. Elise female 58.0 0 0 146.52 B80 C
17569

Graham, Mrs. William Thompson (Edith PC

268 269 1 1 female 58.0 0 1 153.46 C125 S
Junkins) 17582

275 276 1 1 Andrews, Miss. Kornelia Theodosia female 63.0 1 0 13502 77.96 D7 S

Warren, Mrs. Frank Manley (Anna Sophia

366 367 1 1 female 60.0 1 0 110813 75.25 D37 C
Atkinson)

# [Link]

df1 = [Link]()
df1['Cabin_null'] = [Link](df1['Cabin'].isnull(),0,1)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Cabin_null

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S 0

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 0
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S 0

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S 1

Johnson, Mrs. Oscar W

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S 0
(Elisabeth Vilhelmina Berg)

df1 = [Link]()
df1["Bucket"] = [Link](df1["Fare"] < 250, "Low", "High")
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Bucket

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S Low

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S Low
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S Low

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S Low

Johnson, Mrs. Oscar W

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S Low
(Elisabeth Vilhelmina Berg)

df1 = [Link]()

titanic_age_missing_first = df1.sort_values(by='Age',ascending=False, na_position='first')

titanic_age_missing_first.head()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

17 18 1 2 Williams, Mr. Charles Eugene male NaN 0 0 244373 13.00 NaN S

19 20 1 3 Masselmani, Mrs. Fatima female NaN 0 0 2649 7.22 NaN C

26 27 0 3 Emir, Mr. Farred Chehab male NaN 0 0 2631 7.22 NaN C

28 29 1 3 O'Dwyer, Miss. Ellen "Nellie" female NaN 0 0 330959 7.88 NaN Q

29 30 0 3 Todoroff, Mr. Lalio male NaN 0 0 349216 7.90 NaN S

df1 = [Link]()
df1.sort_values(by = 'Age' , ascending = False)[['Name','Ticket','Survived','Pclass', 'Age' ]].head()

Name Ticket Survived Pclass Age

630 Barkworth, Mr. Algernon Henry Wilson 27042 1 1 80.0

851 Svensson, Mr. Johan 347060 0 3 74.0

96 Goldschmidt, Mr. George B PC 17754 0 1 71.0

493 Artagaveytia, Mr. Ramon PC 17609 0 1 71.0

116 Connors, Mr. Patrick 370369 0 3 70.5

Numerical_data = df1.select_dtypes(include=['number'])
Numerical_data.head()
 PassengerId Survived Pclass Age SibSp Parch Fare

0 1 0 3 22.0 1 0 7.25

2 3 1 3 26.0 0 0 7.92

4 5 0 3 35.0 0 0 8.05

6 7 0 1 54.0 0 0 51.86

8 9 1 3 27.0 0 2 11.13

Categorical_data = df1.select_dtypes(include=['object'])
Categorical_data.head()

Name Sex Ticket Cabin Embarked

0 Braund, Mr. Owen Harris male A/5 21171 NaN S

2 Heikkinen, Miss. Laina female STON/O2. 3101282 NaN S

4 Allen, Mr. William Henry male 373450 NaN S

6 McCarthy, Mr. Timothy J male 17463 E46 S

8 Johnson, Mrs. Oscar W (Elisabeth Vilhelmina Berg) female 347742 NaN S

cabin_notna = df1[df1['Cabin'].notna()]
cabin_notna.head()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

10 11 1 3 Sandstrom, Miss. Marguerite Rut female 4.0 1 1 PP 9549 16.70 G6 S

11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 C103 S

21 22 1 2 Beesley, Mr. Lawrence male 34.0 0 0 248698 13.00 D56 S

23 24 1 1 Sloper, Mr. William Thompson male 28.0 0 0 113788 35.50 A6 S

#groupby

df1 = [Link]()
titanic_room = [Link](['Embarked'])['Age'].mean().reset_index()
titanic_room.head()

Embarked Age

0 C 30.76

1 Q 28.09

2 S 29.49

[Link]("Embarked").agg({"Fare": [Link], "Sex": [Link]})

Fare Sex

Embarked

C 59.89 167

Q 13.34 76

S 27.05 642

temp = [Link]("Sex")['Age'].min().to_frame().reset_index()
temp

Sex Age

0 female 0.75

1 male 0.42

[Link](["Embarked", "Pclass"]).agg({"Fare": [[Link], [Link]]})

 Fare

size mean

Embarked Pclass

C 1 84 105.12

2 17 25.36

3 66 11.21

Q 1 2 90.00

2 3 12.35

3 71 11.22

S 1 126 70.50

2 164 20.33

3 352 14.63

[Link](['Survived',"Sex"])['Fare'].first().to_frame()

Fare

Survived Sex

0 female 7.85

male 7.25

1 female 7.92

male 13.00

Tit_groupby = [Link]("Pclass")["Pclass"].count().to_frame()
Tit_groupby

Pclass

Pclass

1 214

2 184

3 489

[Link]('Survived')['Sex'].value_counts().to_frame()

count

Survived Sex

0 male 466

female 81

1 female 231

male 109

[Link](['Survived',"Sex"])['Pclass'].count()/[Link](["Sex"])['Pclass'].count()*100

Survived Sex
0 female 25.96
male 81.04
1 female 74.04
male 18.96
Name: Pclass, dtype: float64

([Link](['Embarked','Pclass']).count()['Fare']/[Link](['Embarked']).count()['Fare'])*100

Embarked Pclass
C 1 50.30
2 10.18
3 39.52
Q 1 2.63
2 3.95
3 93.42
S 1 19.63
2 25.55
3 54.83
Name: Fare, dtype: float64

[Link]("Sex")[["Age","Pclass"]].mean()
 Age Pclass

Sex

female 27.85 2.17

male 30.79 2.39

[Link](["Sex", "Pclass"])["Fare"].mean()

Sex Pclass
female 1 107.08
2 21.97
3 16.12
male 1 67.23
2 19.74
3 12.65
Name: Fare, dtype: float64

titanic_summed = [Link](["Sex", "Embarked"])[["Fare", "Age"]].sum()

titanic_summed

Fare Age

Sex Embarked

female C 5416.11 1691.00

Q 454.86 291.50

S 7811.31 5130.50

male C 4584.90 2276.92

Q 558.94 495.00

S 9553.92 11145.25

[Link](["Embarked", "Pclass"]).agg({"Fare": [[Link], [Link]]})

Fare

size mean

Embarked Pclass

C 1 84 105.12

2 17 25.36

3 66 11.21

Q 1 2 90.00

2 3 12.35

3 71 11.22

S 1 126 70.50

2 164 20.33

3 352 14.63

temp = [Link]("Sex")['Age'].min().reset_index()
temp

Sex Age

0 female 0.75

1 male 0.42

titanic_room= [Link](['Embarked','Sex'])[['Age','Fare']].mean().reset_index()
titanic_room

Embarked Sex Age Fare

0 C female 28.18 75.22

1 C male 33.00 48.26

2 Q female 24.29 12.63

3 Q male 30.94 13.97

4 S female 27.73 38.67

5 S male 30.37 21.71

[Link](['Survived',"Sex","Embarked"])['Pclass'].count().to_frame()
 Pclass

Survived Sex Embarked

0 female C 9

Q 9

S 63

male C 66

Q 37

S 363

1 female C 63

Q 27

S 139

male C 29

Q 3

S 77

[Link]("Embarked").agg({"Fare": [Link], "Sex": [Link]})

Fare Sex

Embarked

C 59.89 167

Q 13.34 76

S 27.05 642

([Link](['Survived',"Sex"])['Fare'].count()/[Link](['Survived'])['Fare'].count()).to_frame()*100

Fare

Survived Sex

0 female 14.81

male 85.19

1 female 67.94

male 32.06

[Link]('Sex')['Embarked'].count().nlargest(2).reset_index()

Sex Embarked

0 male 575

1 female 310

df1[['Pclass', 'Fare']].groupby(['Pclass'], as_index=True).mean()

Fare

Pclass

1 84.36

2 20.66

3 13.67

#pivot_table

quality_pivot = df1.pivot_table(index='Pclass',values='Age', aggfunc=[Link])

quality_pivot

Age

Pclass

1 38.25

2 29.88

3 25.21

x=[Link](pd.pivot_table(df1,index=['Sex','Embarked'],aggfunc='count')['Fare'])
x
 Fare

Sex Embarked

female C 72

Q 36

S 202

male C 95

Q 40

S 440

quality_pivot = df1.pivot_table(index='Pclass',values='Age', aggfunc=[Link])

quality_pivot

Age

Pclass

1 37.0

2 29.0

3 24.0

[Link](df1['Pclass'],df1['Survived'])

Survived 0 1

Pclass

1 80 134

2 97 87

3 370 119

#Cross Tab
[Link](df1['Sex'],df1['Embarked'])

Embarked C Q S

Sex

female 72 36 202

male 95 40 440

#Cross Tab
plot_criteria= ['Sex', 'Pclass']
cm = sns.light_palette("red", as_cmap=True)
(round([Link](df1[plot_criteria[0]], df1[plot_criteria[1]], normalize='columns') * 100,2)).style.background_grad

Pclass 1 2 3

Sex

female 42.990000 41.300000 29.450000

male 57.010000 58.700000 70.550000

[Link](df1['Sex'],df1['Embarked'],normalize = "index" ).style.background_gradient(cmap='crest')

Embarked C Q S

Sex

female 0.232258 0.116129 0.651613

male 0.165217 0.069565 0.765217

plot_criteria= ['Embarked', 'Pclass']

cm = sns.light_palette("red", as_cmap=True)
(round([Link](df1[plot_criteria[0]], df1[plot_criteria[1]], normalize='columns') * 100,2)).style.background_grad

Pclass 1 2 3

Embarked

C 39.620000 9.240000 13.500000

Q 0.940000 1.630000 14.520000

S 59.430000 89.130000 71.980000

[Link](df1['Pclass'], df1['Survived'], margins=True)

Survived 0 1 All

Pclass

1 80 134 214

2 97 87 184

3 370 119 489

All 547 340 887

#Create New Column

df1['sex Titanic map']=df1['Sex'].map({'male':1,'female':0})

[Link]()

sex
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic
map

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S 1

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 0
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S 1

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S 1

Johnson, Mrs. Oscar W

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S 0
(Elisabeth Vilhelmina Berg)

df1["Fare Range"] = [Link](df1["Fare"] < 200, "low", "high")

[Link]()

sex
Fare
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic
Range
map

Braund, Mr. Owen

0 1 0 3 male 22.0 1 0 A/5 21171 7.25 NaN S 1 low
Harris

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 0 low
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S 1 low

McCarthy, Mr. Timothy

6 7 0 1 male 54.0 0 0 17463 51.86 E46 S 1 low
J

Johnson, Mrs. Oscar W

8 9 1 3 (Elisabeth Vilhelmina female 27.0 0 2 347742 11.13 NaN S 0 low
Berg)

df1["age_bins"]= [Link](df1["Age"] ,bins=[1,18,29 , 40 , 50 , 60 , 80] ,labels=["child","teen","adult" , "fortieth"

[Link]()

sex
Fare
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins
Range
map

Braund, Mr.
0 1 0 3 Owen male 22.0 1 0 A/5 21171 7.25 NaN S 1 low teen
Harris

Heikkinen, STON/O2.
2 3 1 3 female 26.0 0 0 7.92 NaN S 0 low teen
Miss. Laina 3101282

Allen, Mr.
4 5 0 3 William male 35.0 0 0 373450 8.05 NaN S 1 low adult
Henry

McCarthy,
6 7 0 1 Mr. Timothy male 54.0 0 0 17463 51.86 E46 S 1 low old
J

Johnson,
Mrs. Oscar
W
8 9 1 3 female 27.0 0 2 347742 11.13 NaN S 0 low teen
(Elisabeth
Vilhelmina
Berg)

df1['is_train'] = [Link](0,1,len(df1)) <=.75

[Link]()
 sex
Fare
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Range
map

Braund,
0 1 0 3 Mr. Owen male 22.0 1 0 A/5 21171 7.25 NaN S 1 low teen False
Harris

Heikkinen,
STON/O2.
2 3 1 3 Miss. female 26.0 0 0 7.92 NaN S 0 low teen True
3101282
Laina

Allen, Mr.
4 5 0 3 William male 35.0 0 0 373450 8.05 NaN S 1 low adult True
Henry

McCarthy,
6 7 0 1 Mr. male 54.0 0 0 17463 51.86 E46 S 1 low old False
Timothy J

Johnson,
Mrs.
Oscar W
8 9 1 3 female 27.0 0 2 347742 11.13 NaN S 0 low teen True
(Elisabeth
Vilhelmina
Berg)

#iloc & loc

[Link][9:12, 2:5] # [Row , Column]

Pclass Name Sex

13 3 Andersson, Mr. Anders Johan male

14 3 Vestrom, Miss. Hulda Amanda Adolfina female

15 2 Hewlett, Mrs. (Mary D Kingcome) female

[Link][2:4, 3:6]

Name Sex Age

4 Allen, Mr. William Henry male 35.0

6 McCarthy, Mr. Timothy J male 54.0

[Link][0:4, 3] = "Anonymous"
[Link]()

sex
Fare
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Range
map

0 1 0 3 Anonymous male 22.0 1 0 A/5 21171 7.25 NaN S 1 low teen False

STON/O2.
2 3 1 3 Anonymous female 26.0 0 0 7.92 NaN S 0 low teen True
3101282

4 5 0 3 Anonymous male 35.0 0 0 373450 8.05 NaN S 1 low adult True

6 7 0 1 Anonymous male 54.0 0 0 17463 51.86 E46 S 1 low old False

Johnson,
Mrs. Oscar
W
8 9 1 3 female 27.0 0 2 347742 11.13 NaN S 0 low teen True
(Elisabeth
Vilhelmina
Berg)

[Link][[3,6,9],[2,3]]

Pclass Name

6 1 Anonymous

10 3 Sandstrom, Miss. Marguerite Rut

13 3 Andersson, Mr. Anders Johan

#Replace
df1["Survived"].replace({0:"Died" , 1:"Saved"} , inplace=True)
[Link]()
 sex
Fare
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Range
map

0 1 Died 3 Anonymous male 22.0 1 0 A/5 21171 7.25 NaN S 1 low teen False

STON/O2.
2 3 Saved 3 Anonymous female 26.0 0 0 7.92 NaN S 0 low teen True
3101282

4 5 Died 3 Anonymous male 35.0 0 0 373450 8.05 NaN S 1 low adult True

6 7 Died 1 Anonymous male 54.0 0 0 17463 51.86 E46 S 1 low old False

Johnson,
Mrs. Oscar
W
8 9 Saved 3 female 27.0 0 2 347742 11.13 NaN S 0 low teen True
(Elisabeth
Vilhelmina
Berg)

#Rename
[Link](columns={"Name" : 'Person Name'},inplace=True)
[Link]()

sex
Person Fare
PassengerId Survived Pclass Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Name Range
map

0 1 Died 3 Anonymous male 22.0 1 0 A/5 21171 7.25 NaN S 1 low teen False

STON/O2.
2 3 Saved 3 Anonymous female 26.0 0 0 7.92 NaN S 0 low teen True
3101282

4 5 Died 3 Anonymous male 35.0 0 0 373450 8.05 NaN S 1 low adult True

6 7 Died 1 Anonymous male 54.0 0 0 17463 51.86 E46 S 1 low old False

Johnson,
Mrs. Oscar
W
8 9 Saved 3 female 27.0 0 2 347742 11.13 NaN S 0 low teen True
(Elisabeth
Vilhelmina
Berg)

# Replace the codes with their full names

df1['Embarked'] = df1['Embarked'].replace({'S': 'Southampton', 'C': 'Cherbourg', 'Q': 'Queenstown'})
[Link](5)

sex
Person Fare
PassengerId Survived Pclass Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Name Range
map

Barber,
Miss.
290 291 Saved 1 female 26.0 0 0 19877 78.85 NaN Southampton 0 low teen False
Ellen
"Nellie"

Graham,
Miss.
887 888 Saved 1 female 19.0 0 0 112053 30.00 B42 Southampton 0 low teen True
Margaret
Edith

Richard,
SC/PARIS
135 136 Died 2 Mr. male 23.0 0 0 15.05 NaN Cherbourg 1 low teen False
2133
Emile

Turpin,
Mrs.
William
41 42 Died 2 John female 27.0 1 0 11668 21.00 NaN Southampton 0 low teen True
Robert
(Dorothy
Ann ...

Calic,
500 501 Died 3 Mr. male 17.0 0 0 315086 8.66 NaN Southampton 1 low child False
Petar

df1['Pclass'][df1['Pclass'] == 1] = 'Rich'
df1['Pclass'][df1['Pclass'] == 2] = 'Middel Class'
df1['Pclass'][df1['Pclass'] == 3] = 'Poor'
[Link]()
 sex
Person Fare
PassengerId Survived Pclass Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Name Range
map

0 1 Died Poor Anonymous male 22.0 1 0 A/5 21171 7.25 NaN Southampton 1 low teen False

STON/O2.
2 3 Saved Poor Anonymous female 26.0 0 0 7.92 NaN Southampton 0 low teen True
3101282

4 5 Died Poor Anonymous male 35.0 0 0 373450 8.05 NaN Southampton 1 low adult True

6 7 Died Rich Anonymous male 54.0 0 0 17463 51.86 E46 Southampton 1 low old False

Johnson,
Mrs. Oscar
W
8 9 Saved Poor female 27.0 0 2 347742 11.13 NaN Southampton 0 low teen True
(Elisabeth
Vilhelmina
Berg)

#Converting Zero Value to NaN Value

[Link][df1['Fare'] == 0,'Fare'] = [Link]
[Link]()

sex
Person Fare
PassengerId Survived Pclass Sex Age SibSp Parch Ticket Fare Cabin Embarked Titanic age_bins is_train
Name Range
map

0 1 Died Poor Anonymous male 22.0 1 0 A/5 21171 7.25 NaN Southampton 1 low teen False

STON/O2.
2 3 Saved Poor Anonymous female 26.0 0 0 7.92 NaN Southampton 0 low teen True
3101282

4 5 Died Poor Anonymous male 35.0 0 0 373450 8.05 NaN Southampton 1 low adult True

6 7 Died Rich Anonymous male 54.0 0 0 17463 51.86 E46 Southampton 1 low old False

Johnson,
Mrs. Oscar
W
8 9 Saved Poor female 27.0 0 2 347742 11.13 NaN Southampton 0 low teen True
(Elisabeth
Vilhelmina
Berg)

# Replace First Three name with any symbol or Value

df1 = [Link]()
[Link][0:2, 'Name'] = '?'
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 ? male 22.0 1 0 A/5 21171 7.25 NaN S

STON/O2.
2 3 1 3 ? female 26.0 0 0 7.92 NaN S
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

Johnson, Mrs. Oscar W (Elisabeth

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
Vilhelmina Berg)

#Replace '?' values in workclass variable with NaN

df1['Name'].replace('?', [Link], inplace=True)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked

0 1 0 3 NaN male 22.0 1 0 A/5 21171 7.25 NaN S

STON/O2.
2 3 1 3 NaN female 26.0 0 0 7.92 NaN S
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S

Johnson, Mrs. Oscar W (Elisabeth

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
Vilhelmina Berg)

#Saving to Excel Format

df1.to_excel('[Link]',sheet_name="Passenger",index=False)

df1 = [Link]()
df1[df1['Age'].isnull()].index

Index([ 17, 19, 26, 28, 29, 31, 32, 36, 42, 45,
...
832, 837, 839, 846, 849, 859, 863, 868, 878, 888],
dtype='int64', length=176)

#Join
#Join
Join = [Link](df1, lsuffix = '_1') # lsuffix = Left Suffix
[Link](2)

PassengerId_1 Survived_1 Pclass_1 Name_1 Sex_1 Age_1 SibSp_1 Parch_1 Ticket_1 Fare_1 Cabin_1 Embarked_1 PassengerId Surv

Braund,
0 1 0 3 Mr. Owen male 22.0 1 0 A/5 21171 7.25 NaN S 1
Harris

Heikkinen,
STON/O2.
2 3 1 3 Miss. female 26.0 0 0 7.92 NaN S 3
3101282
Laina

#Melt
Melt = [Link](df1,id_vars = ['Embarked'], value_vars = ['Survived'])
[Link]()

Embarked variable value

0 S Survived 0

1 S Survived 1

2 S Survived 0

3 S Survived 0

4 S Survived 1

df1 = [Link]()
[Link]

PassengerId int64
Survived int64
Pclass int64
Name object
Sex object
Age float64
SibSp int64
Parch int64
Ticket object
Fare float64
Cabin object
Embarked object
dtype: object

df1["Sex"] = [Link](lambda x:'male' if x==1 else 'female')

df1['Pclass_New'] = df1['Pclass'].apply(lambda x: 'UpperClass' if x == 1 else 0)

display([Link]())
df1['Pclass_New'].value_counts().to_frame()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Pclass_New

0 1 0 3 Braund, Mr. Owen Harris female 22.0 1 0 A/5 21171 7.25 NaN S 0

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 0
3101282

4 5 0 3 Allen, Mr. William Henry female 35.0 0 0 373450 8.05 NaN S 0

6 7 0 1 McCarthy, Mr. Timothy J female 54.0 0 0 17463 51.86 E46 S UpperClass

Johnson, Mrs. Oscar W

8 9 1 3 (Elisabeth Vilhelmina female 27.0 0 2 347742 11.13 NaN S 0
Berg)

count

Pclass_New

0 673

UpperClass 214

df1['Pclass_New'] = df1['Pclass'].apply(lambda x: 5 if x > 2 else 0)

display([Link]())
df1['Pclass_New'].value_counts().to_frame()
 PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Pclass_New

0 1 0 3 Braund, Mr. Owen Harris female 22.0 1 0 A/5 21171 7.25 NaN S 5

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 5
3101282

4 5 0 3 Allen, Mr. William Henry female 35.0 0 0 373450 8.05 NaN S 5

6 7 0 1 McCarthy, Mr. Timothy J female 54.0 0 0 17463 51.86 E46 S 0

Johnson, Mrs. Oscar W

8 9 1 3 (Elisabeth Vilhelmina female 27.0 0 2 347742 11.13 NaN S 5
Berg)

count

Pclass_New

5 489

0 398

[Link]()

['PassengerId',
'Survived',
'Pclass',
'Name',
'Sex',
'Age',
'SibSp',
'Parch',
'Ticket',
'Fare',
'Cabin',
'Embarked',
'Pclass_New']

[Link]()

PassengerId 887
Survived 2
Pclass 3
Name 887
Sex 1
Age 88
SibSp 7
Parch 7
Ticket 679
Fare 246
Cabin 146
Embarked 3
Pclass_New 2
dtype: int64

df1 = [Link]()

women = [Link][df1['Sex'] == 'female']["Survived"]

rate_women = ([Link]()/len(women)).round(3)*100
print("Percentage of women who survived:", rate_women,"%")

men = [Link][df1['Sex'] == 'male']["Survived"]

rate_men = ([Link]()/len(men)).round(3)*100
print("Percentage of men who survived :", rate_men,"%")

Percentage of women who survived: 74.0 %

Percentage of men who survived : 19.0 %

df1 = [Link]()
df1['Age_Range'] = [Link](df1['Age'],
bins=[0.,15,30,45,60,65,[Link]],
labels=[1,2,3,4,5,6])
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Age_Range

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S 2

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S 2
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S 3

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S 4

Johnson, Mrs. Oscar W

8 9 1 3 (Elisabeth Vilhelmina female 27.0 0 2 347742 11.13 NaN S 2
Berg)

df1 = [Link]()
df1['AgeBand'] = [Link](df1['Age'], 5)
df1[['AgeBand', 'Survived']].groupby(['AgeBand'], as_index=False).mean().sort_values(by='AgeBand', ascending=True

AgeBand Survived

0 (0.34, 16.336] 0.56

1 (16.336, 32.252] 0.37

2 (32.252, 48.168] 0.40

3 (48.168, 64.084] 0.43

4 (64.084, 80.0] 0.09

Cabin_Not_NA = df1[df1['Cabin'].notna()]
Cabin_Not_NA.head()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked AgeBand

(48.168,
6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S
64.084]

Sandstrom, Miss. Marguerite PP (0.34,

10 11 1 3 female 4.0 1 1 16.70 G6 S
Rut 9549 16.336]

(48.168,
11 12 1 1 Bonnell, Miss. Elizabeth female 58.0 0 0 113783 26.55 C103 S
64.084]

(32.252,
21 22 1 2 Beesley, Mr. Lawrence male 34.0 0 0 248698 13.00 D56 S
48.168]

(16.336,
23 24 1 1 Sloper, Mr. William Thompson male 28.0 0 0 113788 35.50 A6 S
32.252]

Cabin_NaN = df1[df1['Cabin'].isnull()]
Cabin_NaN.head()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked AgeBand

(16.336,
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S
32.252]

STON/O2. (16.336,
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S
3101282 32.252]

(32.252,
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S
48.168]

Johnson, Mrs. Oscar W (16.336,

8 9 1 3 female 27.0 0 2 347742 11.13 NaN S
(Elisabeth Vilhelmina Berg) 32.252]

Nasser, Mrs. Nicholas (0.34,

9 10 1 2 female 14.0 1 0 237736 30.07 NaN C
(Adele Achem) 16.336]

df1 = pd.get_dummies(df1, columns = ['Embarked'], drop_first=True)

[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin AgeBand Embarked_Q Embarked_S

Braund,
(16.336,
0 1 0 3 Mr. Owen male 22.0 1 0 A/5 21171 7.25 NaN False True
32.252]
Harris

Heikkinen,
STON/O2. (16.336,
2 3 1 3 Miss. female 26.0 0 0 7.92 NaN False True
3101282 32.252]
Laina

Allen, Mr.
(32.252,
4 5 0 3 William male 35.0 0 0 373450 8.05 NaN False True
48.168]
Henry

McCarthy,
(48.168,
6 7 0 1 Mr. male 54.0 0 0 17463 51.86 E46 False True
64.084]
Timothy J

Johnson,
Mrs.
Oscar W (16.336,
8 9 1 3 female 27.0 0 2 347742 11.13 NaN False True
(Elisabeth 32.252]
Vilhelmina
Berg)

df1 = [Link]()

def Grade(Percentage):
if Percentage >= 500:
return 'High'
if Percentage >= 300:
return 'Medium'
if Percentage >= 200:
return 'Average'
if Percentage >= 100:
return 'Low'
if Percentage >= 50:
 return 'VeryLow'
return 'Free'

df1['Fare_Range']=[Link](lambda x: Grade(x['Fare']),axis=1)
[Link]()

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked Fare_Range

0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.25 NaN S Free

STON/O2.
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 7.92 NaN S Free
3101282

4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.05 NaN S Free

6 7 0 1 McCarthy, Mr. Timothy J male 54.0 0 0 17463 51.86 E46 S VeryLow

Johnson, Mrs. Oscar W

8 9 1 3 (Elisabeth Vilhelmina female 27.0 0 2 347742 11.13 NaN S Free
Berg)

wine = pd.read_csv('[Link]')
[Link](3)

fixed volatile citric residual free sulfur total sulfur

chlorides density pH sulphates alcohol quality Id
acidity acidity acid sugar dioxide dioxide

0 7.4 0.70 0.00 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 0

1 7.8 0.88 0.00 2.6 0.10 25.0 67.0 1.0 3.20 0.68 9.8 5 1

2 7.8 0.76 0.04 2.3 0.09 15.0 54.0 1.0 3.26 0.65 9.8 5 2

# Create correlation matrix

corr_matrix = [Link]().abs()
# Select upper triangle of correlation matrix
upper = corr_matrix.where([Link]([Link](corr_matrix.shape), k=1).astype(bool))
# Find features with correlation greater than 0.95
to_drop = [column for column in [Link] if any(upper[column] > 0.30)]
to_drop

['citric acid',
'total sulfur dioxide',
'density',
'pH',
'sulphates',
'alcohol',
'quality',
'Id']

wine['good_quality']=["yes" if x>=7 else 'no' for x in wine['quality']]

[Link]()

fixed volatile citric residual free sulfur total sulfur

chlorides density pH sulphates alcohol quality Id good_quality
acidity acidity acid sugar dioxide dioxide

0 7.4 0.70 0.00 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 0 no

1 7.8 0.88 0.00 2.6 0.10 25.0 67.0 1.0 3.20 0.68 9.8 5 1 no

2 7.8 0.76 0.04 2.3 0.09 15.0 54.0 1.0 3.26 0.65 9.8 5 2 no

3 11.2 0.28 0.56 1.9 0.07 17.0 60.0 1.0 3.16 0.58 9.8 6 3 no

4 7.4 0.70 0.00 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 4 no

#Removing all Negative values

wine['residual sugar'] = wine['residual sugar'].abs()

[Link]()

fixed volatile citric residual free sulfur total sulfur

chlorides density pH sulphates alcohol quality Id good_quality
acidity acidity acid sugar dioxide dioxide

0 7.4 0.70 0.00 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 0 no

1 7.8 0.88 0.00 2.6 0.10 25.0 67.0 1.0 3.20 0.68 9.8 5 1 no

2 7.8 0.76 0.04 2.3 0.09 15.0 54.0 1.0 3.26 0.65 9.8 5 2 no

3 11.2 0.28 0.56 1.9 0.07 17.0 60.0 1.0 3.16 0.58 9.8 6 3 no

4 7.4 0.70 0.00 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 4 no

# Print Row
row_30 = [Link][75]
print(row_30)
fixed acidity 7.8
volatile acidity 0.41
citric acid 0.68
residual sugar 1.7
chlorides 0.47
free sulfur dioxide 18.0
total sulfur dioxide 69.0
density 1.0
pH 3.08
sulphates 1.31
alcohol 9.3
quality 5
Id 106
good_quality no
Name: 75, dtype: object

any_negative_yield = (wine['chlorides'] < 0).any()

if any_negative_yield:
print("The 'chlorides' column contains negative values.")
else:
print("The 'chlorides' column does not contain negative values.")

The 'chlorides' column does not contain negative values.

geo = pd.read_csv('gapminder_full.csv')
[Link](3)

country year population continent life_exp gdp_cap

0 Afghanistan 1952 8425333 Asia 28.80 779.45

1 Afghanistan 1957 9240934 Asia 30.33 820.85

2 Afghanistan 1962 10267083 Asia 32.00 853.10

geo[(geo['population']>10000000) & (geo['country']=='Afghanistan')][['gdp_cap','life_exp','continent']]

gdp_cap life_exp continent

2 853.10 32.00 Asia

3 836.20 34.02 Asia

4 739.98 36.09 Asia

5 786.11 38.44 Asia

6 978.01 39.85 Asia

7 852.40 40.82 Asia

8 649.34 41.67 Asia

9 635.34 41.76 Asia

10 726.73 42.13 Asia

11 974.58 43.83 Asia

reg_medal=[Link](['country','continent']).size().reset_index().head(10)
reg_medal

country continent 0

0 Afghanistan Asia 12

1 Albania Europe 12

2 Algeria Africa 12

3 Angola Africa 12

4 Argentina Americas 12

5 Australia Oceania 12

6 Austria Europe 12

7 Bahrain Asia 12

8 Bangladesh Asia 12

9 Belgium Europe 12

df2=[Link]('country')['continent'].nunique().reset_index()
[Link]()
 country continent

0 Afghanistan 1

1 Albania 1

2 Algeria 1

3 Angola 1

4 Argentina 1

[Link]('country')['continent'].count().nlargest(20).reset_index().head(10)

country continent

0 Afghanistan 12

1 Albania 12

2 Algeria 12

3 Angola 12

4 Argentina 12

5 Australia 12

6 Austria 12

7 Bahrain 12

8 Bangladesh 12

9 Belgium 12

Highest_Population = [Link](10, 'population', keep='all')

Highest_Population.head(10)

country year population continent life_exp gdp_cap

299 China 2007 1318683096 Asia 72.96 4959.11

298 China 2002 1280400000 Asia 72.03 3119.28

297 China 1997 1230075000 Asia 70.43 2289.23

296 China 1992 1164970000 Asia 68.69 1655.78

707 India 2007 1110396331 Asia 64.70 2452.21

295 China 1987 1084035000 Asia 67.27 1378.90

706 India 2002 1034172547 Asia 62.88 1746.77

294 China 1982 1000281000 Asia 65.53 962.42

705 India 1997 959000000 Asia 61.77 1458.82

293 China 1977 943455000 Asia 63.97 741.24

Lowest_Population = [Link](10, 'population', keep='all')

Lowest_Population.head(10)

country year population continent life_exp gdp_cap

1296 Sao Tome and Principe 1952 60011 Africa 46.47 879.58

1297 Sao Tome and Principe 1957 61325 Africa 48.95 860.74

420 Djibouti 1952 63149 Africa 34.81 2669.53

1298 Sao Tome and Principe 1962 65345 Africa 51.89 1071.55

1299 Sao Tome and Principe 1967 70787 Africa 54.42 1384.84

421 Djibouti 1957 71851 Africa 37.33 2864.97

1300 Sao Tome and Principe 1972 76595 Africa 56.48 1532.99

1301 Sao Tome and Principe 1977 86796 Africa 58.55 1737.56

422 Djibouti 1962 89898 Africa 39.69 3020.99

1302 Sao Tome and Principe 1982 98593 Africa 60.35 1890.22

Population = [Link](by = ['country'], axis = 0)['population'].sum()

Range = [Link](Population).reset_index()
Range.sort_values(by= ['population'], ascending = False, inplace = True)
[Link](10)
 country population

24 China 11497920623

58 India 8413568878

134 United States 2738534790

59 Indonesia 1779874000

14 Brazil 1467745520

66 Japan 1341105696

97 Pakistan 1124200629

8 Bangladesh 1089064744

47 Germany 930564520

94 Nigeria 884496214

print(f"\033[031m\033[1m")
print("Unique continent Names :", geo['continent'].nunique())
geo['continent'].value_counts().nlargest(10).to_frame().style.background_gradient(cmap='copper')

Unique continent Names : 5

count

continent

Africa 624

Asia 396

Europe 360

Americas 300

Oceania 24

df1 = geo[geo['continent']=='Asia']
[Link]('country')['life_exp'].max().sort_values(ascending=False).head(5).reset_index()

country life_exp

0 Japan 82.60

1 Hong Kong, China 82.21

2 Israel 80.75

3 Singapore 79.97

4 Korea, Rep. 78.62

[Link]('country')['gdp_cap'].mean().sort_values(ascending=False).index[0:5]

Index(['Kuwait', 'Switzerland', 'Norway', 'United States', 'Canada'], dtype='object', name='country')

[Link]('country')['gdp_cap'].mean().sort_values(ascending=False).head(10)

country
Kuwait 65332.91
Switzerland 27074.33
Norway 26747.31
United States 26261.15
Canada 22410.75
Netherlands 21748.85
Denmark 21671.82
Germany 20556.68
Iceland 20531.42
Austria 20411.92
Name: gdp_cap, dtype: float64

data = geo[geo['continent']=='Africa']
data[data['gdp_cap'] == data['gdp_cap'].max()]['country']

905 Libya
Name: country, dtype: object

data = geo[geo['continent']=='Africa']
data[data['gdp_cap'] == data['gdp_cap'].min()]['country']

334 Congo, Dem. Rep.

Name: country, dtype: object

df1 = geo[(geo['continent'] == 'Europe')]

df1 = df1[df1['gdp_cap'] == df1['gdp_cap'].max()]
df1
from sklearn.model_selection import train_test_split
from [Link] import accuracy_score
from [Link] import LazyClassifier
import [Link] as plt

# Assuming df is your DataFrame containing the Titanic dataset

titanic_data = [Link]()
titanic_data = titanic_data.dropna()

# Select relevant features

features = ['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare']
X = titanic_data[features]
y = titanic_data['Survived']

# Split the data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, r
andom_state=42)

# Create a LazyClassifier
clf = LazyClassifier(verbose=0, ignore_warnings=True, custom_metri
c=None)
models, predictions = [Link](X_train, X_test, y_train, y_test)
print("Models performance:")
models
# If you want to evaluate a specific model (e.g., the best performing
one)
if not [Link]:
best_model = [Link][0]
print(f"\nBest model: {best_model}")

if best_model in [Link]:
best_model_predictions = predictions[best_model]
accuracy = accuracy_score(y_test, best_model_predictions)
print(f"Accuracy of the best model: {accuracy}")
else:
print(f"Warning: Predictions for {best_model} not found in the pr
edictions DataFrame.")
print("Available models in predictions:")
print([Link])
else:
print("No models were successfully trained.")

print("\nShape of predictions DataFrame:", [Link])

print("\nFirst few rows of predictions DataFrame:")
predictions
[Link](figsize=(12, 30))
metrics = ['Accuracy', 'Balanced Accuracy', 'ROC AUC', 'F1 Score']

for i, metric in enumerate(metrics):

[Link](4, 1, i+1)
ax = [Link](x=models[metric], y=[Link], orient='h')
[Link](f'{metric} Comparison')
[Link](f'{metric} (%)')
[Link]('Model')

# Set x-axis to start from 0.40

[Link](0.40, 1.0)

# Add value labels on the bars

for j, v in enumerate(models[metric]):
[Link](v + 0.01, j, f'{v:.3f}', va='center')

plt.tight_layout()
[Link]()
# Assuming 'models' and 'metrics' are already defined DataFrames
summary_df = models[metrics].copy()
summary_df['Model'] = [Link]
summary_df = summary_df.melt(id_vars=['Model'], var_name='Metric'
, value_name='Score')

[Link](figsize=(15, 30))
barplot = [Link](x='Score', y='Model', hue='Metric', data=summ
ary_df, orient='h', palette='viridis')

# Add values at the edge of the bars

for container in [Link]:
barplot.bar_label(container, fmt='%.2f', label_type='edge')

[Link]('Comprehensive Model Performance Comparison')

[Link]('Score (%)')
[Link]('Model')
[Link](title='Metric', bbox_to_anchor=(1.05, 1), loc='upper left')

# Set x-axis to start from 0.50

[Link](0.50, 1.0)
plt.tight_layout()
[Link]()
Prepared By: Syed Afroz Ali (Kaggle Grand Master)

All Types of Data Visualization Kaggle Note Book

Python for Machine Learning Visualization Part 01:

[Link]
Python for Machine Learning Visualization Part 02:

[Link]
import pandas as pd
import numpy as np
import seaborn as sns

import [Link] as px
import plotly.graph_objects as go
import plotly.figure_factory as ff
from [Link] import make_subplots

pd.set_option('[Link]', 2)

# Load the dataset

df = pd.read_csv("heart_disease_uci.csv")
df = [Link]()
[Link](2)

id age sex dataset cp trestbps chol fbs restecg thalch exang oldpeak slope ca thal num

lv fixed
0 1 63 Male Cleveland typical angina 145.0 233.0 True 150.0 False 2.3 downsloping 0.0 0
hypertrophy defect

lv
1 2 67 Male Cleveland asymptomatic 160.0 286.0 False 108.0 True 1.5 flat 3.0 normal 2
hypertrophy

print(f"Records: {[Link][0]}")
print(f"Columns: {[Link][1]}")

Records: 299
Columns: 16

top_leagues = df['cp'].value_counts().nlargest(4).index
display(top_leagues)

[Link](figsize=(15, 6))
[Link](x='age', y='chol', data=df[df['cp'].isin(top_leagues)], hue='cp')
[Link]('Age vs. Cholesterol for Top 4 Chest Pain')
[Link]('Age')
[Link]('Cholesterol')
[Link](title='Chest Pain Type', bbox_to_anchor=(1.05, 1), loc='upper left')
[Link]()

Index(['asymptomatic', 'non-anginal', 'atypical angina', 'typical angina'], dtype='object', name='cp')

import [Link] as px

fig = [Link](df, x='chol', y='age', color='sex')

fig.update_layout(width=1000, height=500)
fig.update_layout(title_text='Scatter Plot of Cholesterol vs. Age (colored by Sex)')

[Link]()

Scatter Plot of Cholesterol vs. Age (colored by Sex)

80

70

60
age

50

40

30

100 200 300 400 500

chol

from [Link] import iplot

fig = [Link](x = df["age"],

labels={"x":"Age"},
title="5-Number-Summary(Box Plot) of Age")
iplot(fig)

5-Number-Summary(Box Plot) of Age

30 40 50 60 70

Age

import [Link] as px

fig = [Link](df, x='chol', y='age', color='cp', size = 'oldpeak', size_max = 30, hover_name = 'exang')

fig.update_layout(width=1000, height=500)

fig.update_layout(title_text='Scatter Plot of Cholesterol vs. Age (colored by cp)')

[Link]()

Scatter Plot of Cholesterol vs. Age (colored by cp)

80
cp

70

60
age

50

40

30

100 200 300 400 500

chol

fig = [Link](data_frame = df,

x="age",
y="chol",
color="cp",
size='ca',
hover_data=['oldpeak'])

fig.update_layout(title_text="<b> Cholesterol Vs Age <b>",

titlefont={'size': 24, 'family':'Serif'},
width=1000,
height=500,
)

[Link]()

Cholesterol Vs Age

cp

500

400
chol

300

200

100

30 40 50 60 70 80

age

import [Link] as px

fig = [Link](df, x='chol', y='age', color='cp', size = 'oldpeak', size_max = 30, hover_name = 'exang',facet_col
fig.update_layout(width=1000, height=500)

fig.update_layout(title_text='Scatter Plot of Cholesterol vs. Age (colored by cp)')

[Link]()

Scatter Plot of Cholesterol vs. Age (colored by cp)

cp=typical angina cp=asymptomatic cp=non-anginal cp=atypical angina
80
cp

70

60
age

50

40

30
100

200

300
400
500
600
700

100

200

300
400
500
600
700

100

200

300
400
500
600
700

100

200

300
400
500
600
700
chol chol chol chol

hover_name='exang' means that the values in the 'exang' column will be shown as tooltips when you hover over the data points
on the scatter plot. This is useful for providing additional information about each data point without cluttering the plot with
labels.

fig=[Link](df,x='age',y='chol',hover_data=['oldpeak'],color='sex',height=400)
[Link]()

sex
Male
4000
Female

3000
chol

2000

1000

0
30 40 50 60 70

age

def generate_rating_df(df):
rating_df = [Link](['cp', 'slope']).agg({'id': 'count'}).reset_index()
rating_df = rating_df[rating_df['id'] != 0]
rating_df.columns = ['cp', 'slope', 'counts']
rating_df = rating_df.sort_values('slope')
return rating_df

rating_df = generate_rating_df(df)
fig = [Link](rating_df, x='cp', y='counts', color='slope')

fig.update_traces(textposition='auto',
textfont_size=20)

fig.update_layout(barmode='stack')
[Link]()

slope
140 downsloping
flat
upsloping
120

100
counts

80

60

40

20

0
asymptomatic atypical angina non-anginal typical angina

cp

def generate_rating_df(df):
rating_df = [Link](['cp', 'slope']).agg({'id': 'count'}).reset_index()
rating_df = rating_df[rating_df['id'] != 0]
rating_df.columns = ['cp', 'slope', 'counts']
rating_df = rating_df.sort_values('slope')
return rating_df

rating_df = generate_rating_df(df)
fig = [Link](rating_df, x='cp', y='counts', color='slope')

fig.update_traces(textposition='auto',
textfont_size=20)

fig.update_layout(barmode='group')

[Link]()

slope
80 downsloping
flat
upsloping
70

60

50
counts

40

30

20

10

0
asymptomatic atypical angina non-anginal typical angina

cp

import [Link] as px
def generate_rating_df(df):
rating_df = [Link](['cp', 'slope']).agg({'id': 'count'}).reset_index()
rating_df = rating_df[rating_df['id'] != 0]
rating_df.columns = ['cp', 'slope', 'counts']
rating_df = rating_df.sort_values('slope')
return rating_df

rating_df = generate_rating_df(df)

fig = [Link](rating_df, x='cp', y='counts', color='slope', barmode='group',

text='counts',
)

fig.update_traces(textposition='auto',
textfont_size=20)

[Link]()

slope
80 84 downsloping
flat
upsloping
70

60

50
counts

49
40 45
36
30
33
20

10
11 5
2 11 3 11 9
0
asymptomatic atypical angina non-anginal typical angina

cp

def generate_rating_df(df):
rating_df = [Link](['cp', 'slope']).agg({'id': 'count'}).reset_index()
rating_df = rating_df[rating_df['id'] != 0]
rating_df.columns = ['cp', 'slope', 'counts']
rating_df = rating_df.sort_values('slope')

# Calculate percentages
total_counts = rating_df['counts'].sum()
rating_df['percentage'] = rating_df['counts'] / total_counts * 100

return rating_df

rating_df = generate_rating_df(df)

fig = [Link](rating_df, x='cp', y='counts', color='slope', text='percentage')

fig.update_traces(
texttemplate='%{text:.1f}%',
textposition='outside',
textfont_size=16
)

fig.update_layout(
barmode='group',
yaxis_title='Count',
xaxis_title='CP',
legend_title='Slope'
)

fig.update_layout(
height=550,
width=1000,
title_text="Distribution of Chest Pain Type by Percentage",
title_font_size=24
)
[Link]()

Distribution of Chest Pain Type by Percentage

28.1% Slope
80

70

60

50
16.4%
15.1%
Count

40
12.0%
11.0%
30

20

3.7% 3.7% 3.7%

10
3.0%
1.7%
0.7% 1.0%
0
asymptomatic atypical angina non-anginal typical angina

CP

fig = [Link](data_frame = df,

x="age",
y="chol",
color="cp",
size='ca',
hover_data=['oldpeak'],
marginal_x="histogram",
marginal_y="box",)

fig.update_layout(title_text="<b> Age vs Cholesterol <b>",

titlefont={'size': 24, 'family':'Serif'},
width=1000,
height=550,
)

[Link]()
 Age vs Cholesterol

cp

500

400
chol

300

200

100

30 40 50 60 70 80

age

fig = [Link](data_frame = df,

x="age",
y="chol",
color="thalch",
size='ca',
hover_data=['oldpeak'],
marginal_x="histogram",
marginal_y="box")

fig.update_layout(title_text="<b> Age vs Cholesterol <b>",

titlefont={'size': 24, 'family':'Serif'},
width=1000,
height=500,
)

[Link]()

Age vs Cholesterol

500

400
chol

300

200

100

30 40 50 60 70 80

age

fig = [Link](data_frame = df,

x="age",
 y="chol",
size ="ca",
size_max=30,
color= "sex",
trendline="ols")
fig.update_layout(title_text="<b> Age vs Cholesterol <b>",
titlefont={'size': 24, 'family':'Serif'},
width=1000,
height=500,
)

[Link]()

Age vs Cholesterol

500

400
chol

300

200

100

30 40 50 60 70 80

age

fig = [Link](data_frame = df,

x="age",
y="chol",
size ="ca",
size_max=30,
color= "sex",
trendline="ols",
trendline_scope="overall",
trendline_color_override="black")

fig.update_layout(title_text="<b>Chest Pain vs Gender<b>",

titlefont={'size': 24, 'family':'Serif'},
width=1000,
height=550,
)
[Link]()
 Chest Pain vs Gender

sex

500

400
chol

300

200

100

30 40 50 60 70 80

age

fig= [Link](df, x='age',height=500,width=900,template='simple_white',

color='sex', # adding categorical column
color_discrete_sequence=['purple','pink'])

fig.update_layout(title={'text':'Histogram of Persons by Age','font':{'size':25}}

,title_font_family="Times New Roman",
title_font_color="darkgrey",

title_x=0.2)

fig.update_layout(
font_family='classic-roman',
font_color= 'grey',
yaxis_title={'text': " count", 'font': {'size':18}},
xaxis_title={'text': " Age", 'font': {'size':18}}
)
[Link]()
 Histogram of Persons by Age
sex
Male
30
Female

25

20
count

15

10

0
30 40 50 60 70

Age

import plotly.graph_objects as go
from [Link] import make_subplots

# Assuming df is your DataFrame

asymptomatic = df[df['cp'] == 'asymptomatic']
non_anginal = df[df['cp'] == 'non-anginal']
atypical_angina = df[df['cp'] == 'atypical angina']
typical_angina = df[df['cp'] == 'typical angina']

fig = make_subplots(rows=2,
cols=2,
specs=[[{'type':'domain'}, {'type':'domain'}],
[{'type':'domain'}, {'type':'domain'}]],
subplot_titles=("Asymptomatic", "Non-Anginal",
"Atypical Angina", "Typical Angina"))

fig.add_trace([Link](labels=asymptomatic["thal"], values=asymptomatic["chol"], name="asymptomatic"), 1, 1)

fig.add_trace([Link](labels=non_anginal["thal"], values=non_anginal["chol"], name="non_anginal"), 1, 2)
fig.add_trace([Link](labels=atypical_angina["thal"], values=atypical_angina["chol"], name="atypical_angina"), 2,
fig.add_trace([Link](labels=typical_angina["thal"], values=typical_angina["chol"], name="typical_angina"), 2, 2)

# Update layout to increase the size of the plot and add main title
fig.update_layout(
height=800,
width=1000,
title_text="Distribution of Cholesterol Levels by Chest Pain Type",
title_font_size=24
)

# Update traces
fig.update_traces(textposition='inside', textfont_size=16)
fig.update_annotations(font_size=20)
[Link]()
 Distribution of Cholesterol Levels by Chest Pain Type
Asymptomatic Non-Anginal

27.5%
38.1%
54.3%
%
2.33
70.2%

3%
7.6

Atypical Angina Typical Angina

17%
3.7
4%
36.1%

56.2%

79.3%

%
68
7.

import [Link] as px

fig = [Link](df, x='chol', y='age', color='cp', size = 'oldpeak', size_max = 30, hover_name = 'exang', range_x
labels = dict(oldpeak = 'oldpeak', chol = 'Cholestrol', age = "Age" ), animation_frame = "chol",

fig.update_layout(width=1000, height=600)

fig.update_layout(title_text='Scatter Plot of Cholesterol vs. Age (colored by cp) with Animation')

[Link]()
 Scatter Plot of Cholesterol vs. Age (colored by cp) with Animation

100
cp

80

60
Age

40

20

0
100 200 300 400 500 600 700 800

Cholestrol

Cholestrol=233.0
▶ ◼

233.0 192.0 283.0 335.0 175.0 216.0 248.0 325.0 182.0 217.0 240.0 277.0 196.0 210.0 319.0 241.0

from [Link] import iplot

gender = df["sex"].value_counts()
display([Link]().to_frame())

fig = [Link](data_frame=gender,
x = [Link],
y = gender,
color=[Link],
text_auto="0.3s",
labels={"y": "Frequency", "index": "Gender"}

)
fig.update_traces(textfont_size=24)

iplot(fig)

count

sex

Male 203

Female 96
 sex
200
203 Male
Female

150
Frequency

100

96.0

50

0
Male Female

sex

from [Link] import iplot

category = df["cp"].value_counts()

fig = [Link](category,
x = [Link],
y = (category / sum(category)) * 100,
color=[Link],
labels={"y" : "Frequency in (Percentage%)", "category":"Category"},
title="Frequency of Chest Pain Category in Percentage",
text = [Link](lambda x: f'{(x / sum(category)) * 100:.1f}%'),
template="plotly_dark"
)

fig.update_layout(showlegend=False)
fig.update_traces(
textfont= {
"family": "consolas",
"size": 20,
}
)

iplot(fig)
 Frequency of Chest Pain Category in Percentage

50

48.2%
40
Frequency in (Percentage%)

30

27.8%
20

16.4%
10

7.7%
0
asymptomatic non-anginal atypical angina typical angina

cp

from [Link] import iplot

ChestPain = df["cp"].value_counts()

fig = [Link](values=ChestPain, names = [Link],

color_discrete_sequence= ["#98EECC", "#FFB6D9", "#99DBF5"],
template="plotly_dark"
)

fig.update_traces(textposition='inside', textfont_size= 20, textinfo='percent+label')

fig.update_layout(showlegend=True,width=1000, height=600)

iplot(fig)

non-anginal
27.8%

asymptomatic
48.2%
a
gin
n
ala
pic .4%
y 16
ngina

at
7.69%
typical a

cp = df["cp"].value_counts()
 fig = [Link](cp,
y = [Link],
x = (cp / sum(cp)) * 100,
color=[Link],
labels={"x" : "Frequency in Percentage(%)", "cp":"Chest Pain"},
orientation="h",
title="Frequency of Chest Pain",
text = [Link](lambda x: f'{(x / sum(cp)) * 100:.1f}%'),
)

fig.update_layout(showlegend=True,width=1000, height=600)

fig.update_traces(
textfont= {
"family": "consolas",
"size": 20
}
)

iplot(fig)

Frequency of Chest Pain

Chest Pai

asymptomatic 48.2%

non-anginal 27.8%
Chest Pain

atypical angina 16.4%

typical angina 7.7%

0 10 20 30 40 50

Frequency in Percentage(%)

fig=[Link]([Link]('cp',as_index=False)['sex'].count().sort_values(by='sex',ascending=False).reset_index(drop
names='cp',values='sex',color='sex',color_discrete_sequence=[Link].Plasma_r,
labels={'cp':'Chest Pain','Sex':'Count'}, template='seaborn',hole=0.4)

fig.update_layout(autosize=False, width=1200, height=700,legend=dict(orientation='v', yanchor='bottom',y=0.40,xanchor

title_x=0.5, showlegend=True)

fig.update_traces(
textfont= {
"family": "consolas",
"size": 20
}
)

[Link]()
 Chest Pain

27.8%

48.2%

16.4%

7.69%

import [Link] as px
from [Link] import iplot
import plotly.graph_objects as go
from [Link] import make_subplots

fig = make_subplots(1,2,subplot_titles=('Age Distribution','Log Age Distribution'))

fig.append_trace([Link](x=df['age'],
name='Age Distribution') ,1,1)

fig.append_trace([Link](x=np.log10(df['age']),
name='Log Age Distribution') ,1,2)

iplot(dict(data=fig))
 Age Distribution Log Age Distribution
45 Age Distribution
30 Log Age Distribution
40

25 35

30
20
25

15 20

15
10

10
5
5

0 0
40 60 1.5 1.6 1.7 1.8 1.9

import numpy as np
import plotly.graph_objs as go
from [Link] import iplot

# Calculate quartiles and IQR

Q25 = [Link](df['chol'], q=0.25)
Q75 = [Link](df['chol'], q=0.75)
IQR = Q75 - Q25
cut_off = IQR * 1.5

# Print number of outliers

print('Number of Cholesterol Lower Outliers:', df[df['chol'] <= (Q25 - cut_off)]['chol'].count())
print('Number of Cholesterol Upper Outliers:', df[df['chol'] >= (Q75 + cut_off)]['chol'].count())

# Group by 'cp' and sort by 'age'

temp = [Link]('cp').sum().sort_values('age', ascending=False)

# Create bar data

data = [
[Link](x=[Link], y=temp['age'], name='Age', text=temp['age'], textposition='auto'),
[Link](x=[Link], y=temp['chol'], name='Cholesterol', text=temp['chol'], textposition='auto')
]

# Define layout
layout = [Link](
xaxis=dict(title='Chest Pain', titlefont=dict(size=25)),
yaxis=dict(title='Values', titlefont=dict(size=25)),
showlegend=True,
width=1300,
height=600
)

# Create figure and plot

fig = [Link](data=data, layout=layout)
iplot(fig)

Number of Cholesterol Lower Outliers: 1

Number of Cholesterol Upper Outliers: 5
 35k 35949

30k

25k
Values

20k
20367

15k

12019
10k

8032
5k
4475
2510
0
asymptomatic non-anginal atypical angina

Chest Pain

import plotly.graph_objs as go
from [Link] import iplot

# Assuming df is your DataFrame

top_03_cp = [Link]('cp').sum()['age'].sort_values(ascending=False)[0:3]
top_03_AGE = [Link](by='cp').sum().sort_values(by='age', ascending=False)[0:3]['chol']

data = [
[Link](
x=top_03_cp.index,
y=top_03_cp,
name='Top 3 age',
text=top_03_cp,
textposition='auto'
),
[Link](
x=top_03_AGE.index,
y=top_03_AGE,
name='Top 3 cholesterol',
text=top_03_AGE,
textposition='auto'
)
]

layout = [Link](
title="Grouped Bar Plot For Age and Cholesterol<br>(For The Top Three types of Chest Pain)",
barmode='group'
)

iplot(dict(data=data, layout=layout))
 Grouped Bar Plot For Age and Cholesterol
(For The Top Three types of Chest Pain)

Top 3 age
35k 35949 Top 3 cholesterol

30k

25k

20k
20367

15k

12019
10k

8032
5k
4475
2510
0
asymptomatic non-anginal atypical angina

gap_df = pd.read_csv("gapminder_full.csv")

display(gap_df.head(2))

fig = [Link](data_frame=gap_df,
x="continent",
y="population",
color="continent",
animation_frame="year",
animation_group="country",
range_y=[0,4000000000])
[Link]()

country year population continent life_exp gdp_cap

0 Afghanistan 1952 8425333 Asia 28.80 779.45

1 Afghanistan 1957 9240934 Asia 30.33 820.85

4B
continent
3.5B Asia
Europe
Africa
3B
Americas
Oceania
2.5B
population

2B

1.5B

1B

0.5B

0
Asia Europe Africa Americas Oceania

continent

year=1952
▶ ◼

1952 1957 1962 1967 1972 1977 1982 1987 1992 1997 2002 2007

fig = [Link](gap_df,x='gdp_cap',y='life_exp',color='continent',size='population',size_max=60,hover_name="country"
animation_frame="year",animation_group='country',log_x=True,range_x=[100,100000],range_y=[25,90],
 labels=dict(Population ="Populations",gdp_cap="Gdp Per Capital",life_exp="Life Expentacy"))

fig.update_layout(
height=550,
width=1500,
title_text="Distribution of GDP Cap Vs Life Expentacy",
title_font_size=24
)

[Link]()

Distribution of GDP Cap Vs Life Expentacy

90

80

70
Life Expentacy

60

50

40

30

2 3 4 5 6 7 8 9 2 3 4 5 6 7
100 1000

Gdp Per Capital

year=1952
▶ ◼

1952 1957 1962 1967 1972 1977 1982

#Grouping the data by state

df1 = df[['cp','age','chol','num']]
[Link]('cp').sum().head(10).style.background_gradient(cmap='Blues')

age chol num

cp

asymptomatic 8032 35949.000000 225

atypical angina 2510 12019.000000 14

non-anginal 4475 20367.000000 33

typical angina 1285 5454.000000 11

import pandas as pd
import [Link] as px

grouped_df = [Link](['cp', 'thal']).size().reset_index(name='count')

fig = [Link](grouped_df,
y="cp",
x='count',
color='thal',
title='Count of Passengers by cp and thal',
labels={'count': 'Number of Patients'},
text_auto=True)
[Link]()
 Count of Passengers by cp and thal

thal
fixed defect
typical angina 2 13 8
normal
reversable defect

non-anginal 2 59 22
cp

atypical angina 2 39 8

asymptomatic 12 53 79

0 50 100 150

Number of Patients

# color palette for visualizations

import [Link] as plt

colors = ['#2B2E4A', '#E84545', '#903749', '#53354A',]

palette = sns.color_palette( palette = colors)

[Link](palette, size = 2.5)

[Link](-0.5,
-0.7,
'Color Palette',
{'font':'monospace',
'size': 24,
'weight':'normal'}
)

[Link]()

def format_title(title, subtitle=None, subtitle_font=None, subtitle_font_size=None):

title = f'<b>{title}</b>'
if not subtitle:
return title
subtitle = f'<span style="font-family: {subtitle_font}; font-size: {subtitle_font_size}px;">{subtitle}</span>'
return f'{title}<br>{subtitle}'

import plotly.figure_factory as ff

_ = [Link](['cp', 'thal']).[Link]().unstack()
z = _.[Link]()
x = _.[Link]()
y = _.[Link]()

fig = ff.create_annotated_heatmap(z = z,
x = x,
y = y,
 xgap = 3,
ygap = 3,
colorscale = ['#53354A', '#E84545']
)

title = format_title('cp',
'thal.',
'Chol',
12
)

fig.update_layout(title_text = title,
title_x = 0.5,
titlefont={'size': 24,
'family': 'Proxima Nova',
},
template='plotly_dark',
paper_bgcolor='#2B2E4A',
plot_bgcolor='#2B2E4A',

xaxis = {'side': 'bottom'},

xaxis_showgrid = False,
yaxis_showgrid = False,
yaxis_autorange = 'reversed',
)

[Link]()

cp
thal.

asymptomatic 12 53 79

atypical angina 2 39 8

non-anginal 2 59 22

typical angina 2 13 8

fixed defect normal reversable defect

# available templates
template = ['ggplot2','plotly_dark', 'seaborn', 'simple_white', 'plotly']

fig = [Link](df,
x="cp",
y=None,
color="sex",
width=1200,
height=450,
histnorm='percent',
color_discrete_map={
"male": "RebeccaPurple", "female": "lightsalmon"
},
template="plotly_dark"
)

fig.update_layout(title="Gender Chest Pain",

font_family="San Serif",
bargap=0.2,
barmode='group',
titlefont={'size': 24},
legend=dict(
orientation="v", y=1, yanchor="top", x=1.25, xanchor="right")
)
[Link]()
 Gender Chest Pain

50

percent 40

30

20

10

0
typical angina asymptomatic non-anginal atypical angina

cp

from [Link] import make_subplots

# data students performance

fig = make_subplots(rows=1, cols=2,
specs=[[{'type':'domain'}, {'type':'domain'}],
])
fig.add_trace(
[Link](
labels=df['cp'],
title="Chest Pain",
titlefont={'size':20, 'family': 'Serif',},
values=None,
hole=0.85,
), col=1, row=1,
)
fig.update_traces(
hoverinfo='label+value',
textinfo='label+percent',
textfont_size=12,
)

fig.add_trace(
[Link](
labels=df['cp'],
title="Chest Pain",
titlefont={'size':20, 'family': 'Serif',},
values=None,
hole=0.5,
), col=2, row=1,
)
fig.update_traces(
hoverinfo='label+value',
textinfo='label+percent',
textfont_size=12,
)
[Link](title="<b> Heart Disesse <b>",
titlefont={'size':20, 'family': 'Serif',},
showlegend=False,
height=600,
width=1000,
template=None,
)

[Link]()
 Heart Disesse

non-anginal
27.8%
non-anginal
27.8%

asymptomatic asymptomatic
Chest Pain 48.2%
Chest Pain 48.2%

at
yp 16
ic .4%
al
an

ngina
gi
atypical angina

na

7.69%
16.4%

typical a
typical angina
7.69%

from [Link] import make_subplots

# data titanic
fig = make_subplots(rows=1, cols=2,
specs=[[{'type':'domain'}, {'type':'domain'}],
])
fig.add_trace(
[Link](
labels=df['cp'],
values=None,
hole=.4,
title='Chest Pain',
titlefont={'color':None, 'size': 24},

),
row=1,col=1
)
fig.update_traces(
hoverinfo='label+value',
textinfo='label+percent',
textfont_size=12,
marker=dict(
colors=['lightgray', 'lightseagreen'],
line=dict(color='#000000',
width=2)
)
)

fig.add_trace(
[Link](
labels=df['sex'],
values=None,
hole=.4,
title='Sex',
titlefont={'color':None, 'size': 24},
),
row=1,col=2
)
fig.update_traces(
hoverinfo='label+value',
textinfo='label+percent',
textfont_size=16,
marker=dict(
colors=['lightgray', 'lightseagreen'],
line=dict(color='#000000',
width=2)
)
)
[Link](title="<b> Heart Desies <b>",
 titlefont={'color':None, 'size': 24, 'family': 'San-Serif'},
showlegend=False,
height=600,
width=950,
)
[Link]()

Heart Desies

non-anginal
27.8% Female
32.1%

asymptomatic
48.2%
Chest Pain Sex
at

Male
yp 16

67.9%
ic .4%
al
an
gi
na

typical angina
7.69%

# data students performance

fig = [Link](df,
path=['cp', 'sex'])
fig.update_layout(title_text="<b>Chest Pain vs Gender<b>",
titlefont={'size': 24, 'family':'Serif'},
width=750,
height=750,
)
[Link]()
 Chest Pain vs Gender

Male

Female

asymptomatic

typic Female
al an
gina

non-anginal
Male
Male atypical angina

Female

Female Male

fig = [Link](df, x="cp",

width=600,
height=400,
histnorm='percent',
category_orders={
"cp": ["asymptomatic", "non-anginal", "atypical angina", "typical angina"],
"sex": ["Male", "Female"]
},
color_discrete_map={
"Male": "RebeccaPurple", "Female": "lightsalmon",
},
template="simple_white"
)

fig.update_layout(title="Chest Pain Type",

font_family="San Serif",
titlefont={'size': 20},
legend=dict(
orientation="v", y=1, yanchor="top", x=1.0, xanchor="right" )
).update_xaxes(categoryorder='total descending')
# custom color
colors = ['gray',] * 4
colors[3] = 'crimson'
colors[0] = 'lightseagreen'

fig.update_traces(marker_color=colors, marker_line_color=None,
marker_line_width=2.5, opacity=None)
[Link]()
 Chest Pain Type
50

40

30
percent

20

10

0
asymptomatic non-anginal atypical angina typical angina

cp

fig = [Link](df, x="cp",

width=600,
height=500,
histnorm='percent',
template="simple_white",
)
fig.update_layout(title="Types of Chest Pain",
font_family="San Serif",
titlefont={'size': 20},
showlegend=True,
legend=dict(
orientation="v",
y=1.0,
yanchor="top",
x=1.0,
xanchor="right"
)
)
fig.update_traces(marker_color=None, marker_line_color='white',
marker_line_width=1.5, opacity=0.99)
[Link]()

Types of Chest Pain

50

40

30
percent

20

10

0
typical angina asymptomatic non-anginal atypical angina

cp

colors = ['rgba(38, 24, 74, 0.8)', 'rgba(71, 58, 131, 0.8)',

'rgba(122, 120, 168, 0.8)', 'rgba(164, 163, 204, 0.85)',
'rgba(190, 192, 213, 1)']

data = df[['sex']]

fig = [Link](df,
y="sex",
 orientation='h',
width=800,
height=350,
histnorm='percent',
template="plotly_dark"
)
fig.update_layout(title="<b>Heart Disease<b>",
font_family="San Serif",
bargap=0.2,
barmode='group',
titlefont={'size': 28},
paper_bgcolor='lightgray',
plot_bgcolor='lightgray',
legend=dict(
orientation="v",
y=1,
yanchor="top",
x=1.250,
xanchor="right",)
)
annotations = []
[Link](dict(xref='paper', yref='paper',
x=0.0, y=1.2,
text='Heart Disease',
font=dict(family='Arial', size=16, color=colors[2]),
showarrow=False))
[Link](dict(xref='paper', yref='paper',
x=0.50, y=0.85,
text='30.4%',
font=dict(family='Arial', size=20, color=colors[2]),
showarrow=False))
[Link](dict(xref='paper', yref='paper',
x=1.08, y=0.19,
text='69.6%',
font=dict(family='Arial', size=20, color=colors[2]),
showarrow=False))

fig.update_layout(
autosize=False,
width=800,
height=350,
margin=dict(
l=50,
r=50,
b=50,
t=120,
),
)

fig.update_layout(annotations=annotations)
fig.update_xaxes(showgrid=False)
fig.update_yaxes(showgrid=False)
[Link]()

Heart Disease
Heart Disease

Female 30.4%
sex

Male 69.6%

0 10 20 30 40 50 60 70

percent

# Plotting the pie chart

[Link](figsize=(20, 5))

# Pie chart
[Link](1, 2, 1)
quality_counts = df['cp'].value_counts()
[Link](quality_counts, labels=quality_counts.index, colors=sns.color_palette('PuBuGn', len(quality_counts)), autopct
[Link]('Chest Pain Distribution')

# Count plot
[Link](1, 2, 2)
ax = [Link](data=df, x='cp',palette='PuBuGn')
# Add count values above each bar
for i in range(len([Link])):
ax.bar_label([Link][i], label_type='edge')

[Link]('Chest Pain Distribution')

[Link]('Chest Pain')
[Link]('Count')
plt.tight_layout()
[Link]()

[Link](figsize=(20, 5))

for i, col in enumerate(['age', 'chol', 'oldpeak'], 1):

[Link](1, 3, i)
ax = [Link](x='sex', y=col, data=df)
[Link](f'{col} Comparison')
[Link](col if i == 1 else '')

# Add count values above each bar

for i in range(len([Link])):
ax.bar_label([Link][i], label_type='edge')

[Link]()

[Link](df[['cp','age','chol','thalch']], hue='cp', aspect=1.5,dropna=True,palette='bright')

[Link]()
# Group by quality and calculate the mean for each quality
grouped_mean = df[['cp','age','trestbps','chol','thalch']].groupby('cp').mean().round(2)

[Link](figsize=(20, 6))

# Plot the grouped bars using Seaborn's barplot

ax = [Link](data=grouped_mean.reset_index().melt(id_vars='cp'),
x='variable', y='value', hue='cp', palette='CMRmap', alpha=0.8)

# Add count values above each bar

for i in range(len([Link])):
ax.bar_label([Link][i], label_type='edge')

[Link]('Features')
[Link]('Mean Value')
[Link]('Grouped Barplot by Chest Pain type')

# Rotate x-axis labels

[Link](rotation=45, ha='right')

[Link](title='Chest Pain')
[Link]()

# Visualization 8: Violin Plot - Skill Moves Distribution

[Link](figsize=(12, 6))
[Link](x='cp', y='chol', data=df)
[Link]('Distribution of Chest Pain with Cholesterol ')
[Link]('Chest Pain Type')
[Link]('Cholesterol ')
[Link]()
import pandas as pd
import [Link] as plt
import numpy as np

# Assuming 'df' is your DataFrame

cp_attributes_comparison = [Link][df['cp'].isin(['asymptomatic', 'non-anginal', 'atypical angina','typical angina'
attributes_to_compare = ['age', 'trestbps', 'chol', 'thalch', 'oldpeak', 'ca']

fig, ax = [Link](figsize=(10, 10), subplot_kw=dict(polar=True))

for cp in cp_attributes_comparison['cp'].unique():
cp_data = cp_attributes_comparison.loc[cp_attributes_comparison['cp'] == cp]

# Calculate mean values for each attribute

values = cp_data[attributes_to_compare].mean().[Link]().tolist()
values += values[:1] # Close the circle for radar plot

angles = [n / float(len(attributes_to_compare)) * 2 * [Link] for n in range(len(attributes_to_compare))]

angles += angles[:1]

[Link](angles, values, linewidth=2, linestyle='solid', label=cp)

[Link](angles, values, alpha=0.25)

# Set the labels

ax.set_xticks(angles[:-1])
ax.set_xticklabels(attributes_to_compare)

# Add legend
[Link](loc='upper right', bbox_to_anchor=(1.3, 1))

# Add title
[Link]('Chest Pain Attributes Comparison')

# Show the plot

plt.tight_layout()
[Link]()
jobs = pd.read_csv("jobstreet_all_job_dataset.csv")
jobs = [Link](5000)
jobs = [Link](columns=['job_id'], axis=1)
jobs = jobs.reset_index()
jobs = [Link](columns=['index'], axis=1)
display([Link])
[Link](2)

(5000, 10)
job_title company descriptions location category subcategory role type salary listingDate

RM 3,000
AESD JOB –
MARKETING Marketing & Marketing marketing- Full 2024-03-
0 INTERNATIONAL DESCRIPTIONS\nWork Petaling RM 4,000
EXECUTIVE Communications Assistants/Coordinators executive time 21T08:08:18Z
(M) SDN. BHD. closely with the sales ... per
month

RM 2,500
Job –
E-Commerce JOBSGURU Administration & Client & Sales sales- Full 2024-05-
1 Description\nPerform Petaling RM 3,500
Sales Admin SDN. BHD. Office Support Administration administration time 24T12:59:40Z
CS activities by repl... per
month

import missingno as msno

# Create a figure with two subplots arranged in a 1x2 grid

fig, axes = [Link](nrows=1, ncols=2, figsize=(20, 6))

# Plot the original DataFrame with missing values

[Link](jobs, ax=axes[0])
axes[0].set_title("Original DataFrame with Missing Values",fontsize=24,color='Red')

# Drop rows with missing values and plot the resulting DataFrame
job = [Link]()

[Link](job, ax=axes[1])
axes[1].set_title("DataFrame after Dropping Missing Values",fontsize=24,color='Green')

plt.tight_layout()
[Link]()
import re

def clean_and_calculate_mean(salary):
try:
# Remove currency symbols, words, and extra characters
salary = [Link]('RM', '').replace('MYR', '').replace('$', '').replace('per month', '').replace('p.m.'

# Handle ranges with different separators

if '–' in salary:
salary_range = [Link]('–')
elif '-' in salary:
salary_range = [Link]('-')
elif '—' in salary:
salary_range = [Link]('—')
else:
salary_range = [salary]

# Convert values to integers, handling potential errors

salary_values = []
for value in salary_range:
try:
value = int(float([Link](',', '').strip()))
salary_values.append(value)
except ValueError:
pass # Ignore non-numeric values

# Calculate mean if at least two valid values are found

if len(salary_values) >= 2:
salary_mean = sum(salary_values) / len(salary_values)
return salary_mean
else:
return None

except Exception as e:
print(f"Error processing salary '{salary}': {e}")
return None

# Apply the function to the salary column

job['Salary'] = job['salary'].apply(clean_and_calculate_mean)
job = [Link]('salary',axis=1)
[Link](2)

job_title company descriptions location category subcategory role type listingDate Salary

AESD JOB
MARKETING Marketing & Marketing marketing- Full 2024-03-
0 INTERNATIONAL DESCRIPTIONS\nWork Petaling 3500.0
EXECUTIVE Communications Assistants/Coordinators executive time 21T08:08:18Z
(M) SDN. BHD. closely with the sales ...

Job
E-Commerce JOBSGURU Administration & Client & Sales sales- Full 2024-05-
1 Description\nPerform Petaling 3000.0
Sales Admin SDN. BHD. Office Support Administration administration time 24T12:59:40Z
CS activities by repl...

import [Link] as px
from [Link] import init_notebook_mode
init_notebook_mode(connected=True)

jobType = job['type'].value_counts()

fig = [Link](values=[Link], names=[Link](), color=[Link](),color_discrete_sequenc

fig.update_layout(width=1000, height=800)

fig.update_traces(textfont_size=20)

fig.update_traces(pull=[0.1, 0.1, 0.2], textposition='outside')

# Set layout properties

fig.update_layout(margin = dict(t=50, l=10, r=10, b=25),
title='Employment Types in the Job Market of Malaysia',
title_x=0.5,
 title_y=0.98)

[Link]()

Employment Types in the Job Market of Malaysia

4.63%
0.408%

0.0454%

94.9%

top_n = 50

filtered_data = job['job_title'].value_counts().head(top_n).reset_index()
filtered_data.columns = ['job_title', 'count']

fig = [Link](filtered_data, path=[[Link]('all'), 'job_title'], values='count')

fig.update_traces(root_color='lightgrey')

fig.update_traces(textfont_size=16)

fig.update_layout(width=1000, height=600)
fig.update_layout(margin=dict(t=50, l=25, r=25, b=25),
title='Top Job Openings: Job Roles in Malaysia',
title_x=0.5,
title_y=0.98)

[Link]()
 Top Job Openings: Job Roles in Malaysia

all
Business Development Executive Senior Account Executive Account Assistant
Marketing Executive
Human Resource Executive

Account Executive Purchasing Executive

Senior Marketing Executive Administrative Assistant Customer Service Executive E-Commerce Executive
HR Executive
Finance Executive Project Engineer

Business Development Human Resources Executive

Personal Assistant HR cum Admin Executive
IT Executive
Accounts Assistant

Admin Assistant
Admin Executive ACCOUNT EXECUTIVE
Finance Manager Sales Admin Executive
Customer Service Mechanical Engineer SALES EXECUTIVE
Sales Admin

HR & Admin Executive PURCHASING ASSISTANT

Graphic Designer Sales Manager Software Engineer

Account Clerk

Sales Executive Accounts Executive Accountant

Project Manager HR Manager Production Engineer Senior Finance Executive

Audit Associate Business Development Manager

[Link]('Kuala Lumpur Sentral','Kuala Lumpur', inplace=True)

[Link]('Bangsar South', 'Bangsar', inplace=True)
[Link]('Klang District', 'Klang/Port Klang', inplace=True)
[Link]('Penang Island', 'Penang', inplace=True)
job['location'] = job['location'].[Link](' District', '')

top_location = ['Kuala Lumpur', 'Petaling', 'Penang']

filtered_data = job[job['location'].isin(top_location)]

job_title_counts = filtered_data['job_title'].value_counts()

top_n = 10
top_job_titles = job_title_counts.head(top_n).[Link]()

filtered_data = filtered_data[filtered_data['job_title'].isin(top_job_titles)]

fig = [Link](filtered_data, path=[[Link]('all'), 'location', 'job_title'])

fig.update_traces(root_color='lightgrey')
fig.update_layout(width=1000, height=600)
fig.update_layout(margin=dict(t=50, l=25, r=25, b=25),
title='Top Job Opportunities in Kuala Lumpur, Petaling, and Penang',
title_x=0.5,
title_y=0.98)
[Link]()
 Top Job Opportunities in Kuala Lumpur, Petaling, and Penang

all

Petaling Kuala Lumpur

Sales Executive Account Assistant Marketing Executive Account Executive Admin Assistant Finance Executive

HR Assistant Sales Executive

Customer Service Executive

Account Executive Customer Service Executive
Accounts Executive

Marketing Executive
Accounts Executive
Business Development Executive
Finance Executive Business Development Executiv

Admin Assistant
Penang
Business Development Executive
Account Assistant Admin Assistant
Account Executive Sales Executive
HR Assistant

# plot a sunburst chart

fig = [Link](job, path=['location','category'])

# configurate the plot layout

fig.update_layout(
margin=dict(t=50, l=25, r=25, b=25),
width=900, # Set the width of the plot
height=800, # Set the height of the plot
title='Job Vacancies Available Across Malaysia by Category',
title_x=0.48,
title_y=0.98
)

[Link]()
 Job Vacancies Available Across Malaysia by Category

Defencey
Governm & Technolog
Administr

perty

l
ent &
y & Tourism
g & Strategy

Lega
ent
Science

& Developm

n
ng
Trad Superann Media
& Pro
Hospitalit

uatio
Consultin

rtising n & Traini

es
Info

ity Services

ical
ervic

tion
Estate

s&
Commun

ts
Med
, Art
tio

c
Accounting

rma

En & Arch l Service s

Real

S
Educa

u
Sales

gin itect s
ruc

istic
rod
re &
es &

rin e
Adve
Ma

e&

ur
nst

Log
hca
tion

rP

g
ranc

Co
lt
ation & O

me
&
Insu
rke

Hea

ice
t
cia
& Co

por
En

Ba uring, onsu

ee
sig inan

rv
ing Trans
tin
Hu

gin

s
&C

Se
mm

&F

n
m

g&
M

tio
n
ee
an

ail

er
an

unic
Ca

Ret

t
ica
uf
Re

om

en
rin

e
ffice Supp
Re

t
nk
ll

fac
Co

D
ac
t

ation
ai

Ce

itm
C

un
so

nu
l&

t
Ed on

st
ur
nt

Ma
s C

ur
Ba uca tru

m
Cu

u
in
o

re
ct nsu

cr
mu
ti

ce
g
De nk

Tech
sig ing on

m
i

&

, T om
& on m

Re
Sales

&
n &
Ad Rea
er

Cu

Co
He vert Esta &
Ar ina Tra
F

ra
l

&

nic
a isin
Pr

tre
ch nc in

nolo

&
Ho lth te

st

ns r S
ca g &
re , Art Pro
od
sp

i
Le
ite ia
ita

ct l S ng
ga

Re
l lit
Sc y & s
M & pert

ort
ienc &

&

s
uc
To

po er
Con

ism ed M
Spo sult Tra
e ur
ica ed y ur e
rt ing des
& & &
Rec & Stra
Te

ce
en
Ser

e rvic

ati
ia
reat
tegyvice
ch
l
ts
ion s
no

cr ogi

gy
logy

rt vic

an ting
es

rt

ur
ui

M all C
&
po

on

so
tm tics
up

e
Re
s
Ac S

k
C
e
e

nt
co

ar
un fic
tin Of

um
Adm g n&
tio

H
inis
tra
tion i stra nolo
gy
Petaling n ech
Eng &O dmi icatio
nT
inee ffic A mm
un
ring e Sup &C
o
Man por tion
ufactu t Kuala Lumpur rma
Trans
ring, Info
Marketin port
g & Co
mmun & Log
Sales ications istics
Human Reso Johor Bahru Accounting
urces & Recr
uitment
Information & Comm
unication Techn
Real Estate & Property ology
Retail & Consumer Products
Construction
Call Centre & Customer Service Retail & Consumer Products
Banking & Financial Services
Construction
Administration & Office Support

Design & Architecture

Human Resources & Recruitment
Retail & Consumer Products
Healthcare & Medical
Manufacturing, Transport & Logistics

Education & Training

Administration & Office Support
Administration & Office Support
Human Resources & Recruitment
Administration & Office Support

Banking & Financial Services

Administration & Office
Information & Communication Support
Technology
Mining, Resources
Manufacturing, Transport & Energy
Healthcare & Medical Human Resources &
& Logistics
Accounting
Recruitment
Legal Wangsa Maju
Tangkak Accounting
Engineering
Taman Desa
Science & Technology Taman Connaught Accounting
& Media Engineering
Advertising, Arts Sri Petaling
CEO & General Management
& Conservation Sentul Accounting
Farming, Animals Sarawak Administration Construction

g
& Office Support
Hospitality & Tourism Muar

Engineerin
& Energy Engineering
Mining, Resources Maluri
Malaysia Engineering
Healthcare &

ng
Kudat Medical
Kota Bharu Administration Accounting

Penang
& Office Support
Jasin Retail & Consumer
Bandar Bentong Products
Sri Permaisur Marketing & Accounting
Taman Ampang i Communications
Tun Dr Information Accounting
Ismail & Communication
Technology

ba
Sandakan
Sales
Manufacturing, Accounting
Pontian Transport
Pahang Healthcare & Logistics
Mid Valley & Medical
Kampun City Accounting
g Malaysi Kedah

ting
Sales

Su
a Raya Human Resources Accounting
Sibu &
Division Education Recruitment
Sabak & Training

Accoun
Bernam Administratio Sales
Negeri Accounting
Sembil n & Office
Manufacturin Support
an Accounting
g, Transport

/
Kucha
i Lama & Logistics
Bukit
Daman Design Accounting
Manufacturi & Architecture
sara Accounting
Bukit ng, Transport
Bintan Healthcare & Logistics
g Retail &
Consumer& Medical

m
Mir Per Human
Resources Products
i Div ak
Marketing & Recruitmen
Call Centre & Communica
Manufactu t
isio & Customer
Human ring, Transport
tions

Ala
Sabah n
Service
Resources & Logistics
Insurance & Recruitme
& Superannu

logy
nt
Kuan Engineerin ation

Ko
g
tan
chno
Ba Sales

Sales
ndar ta Se
Advertisin Accountin
g, Arts g

n Te
Marketing & Media
tar Accountin

s
Ma & Communi g

gistic
CEO & Healthcar

icatio
Mo lay Manufact General
cations
e & Medical

nt sia Retail
uring,
TransporManagem

ah
mmun
& Consume ent
t & Logistics
Kia Manufact

t & Lo
r
uring, Products

ther O ra Transpor

& Co
t & Logistics
Farming Engineer

spor
, Animals Accountiing

i
g
ation Ku K
& Conserv ng

ala Buki ulim

Human
Sales
ation

Sh
Adminis

, Tran
Resourc

Inform
ra
tration

t
es &
& Office Recruitm

t Ja
ppor
Support ent

S
turing Office Su tment
Ku ela lil
Sales
or

Klan
Marketi Account

n
Retail

a
ng &
Commuing

Bin Kua la M gor

Informa & Consum

ac
tion er Product nication
& Commu

uf
s
nications

ng
&

Kuala Lum
tu la L
s
Admini Consult

Man ion crui u

stration ing Techno
Hospita
an da
Pe
& Strateg logy

KL
& Office

istic
lu
& Re ns
y

istrat
AccounSuppor
lity & Accoun ting t
ng

s M D g Tourism ting
in ce io Ec ivisio at
Adm ela
Retail

Log
sour icat
& Consum

o
Inform Sales

gi/Serda
er
ationMarket EngineProduc

an Re mun C n
Admin

M
Constr ering ts

ka
& Comm ing
istratio & Comm

Ku
uction

ity
n& unicati unicati

Com Service T ela

Office on Technoons

rt &
Suppo

Hum
Manuf Retail logy
la

Inform rt
ationacturin & Consu

g& Ke eng ka

ch
& Comm g, Transp mer Sales
ng

unicat ort & Produc

er
Call

ketin Custom
spo
Centre Bankin Hospit ion Logist ts
& Custo g & Financ ality Techn ics
& Tourisology

in
ah
mer ial Servic m

Mar ntre & inin

rt
Manuf

p
acturi Real Servic
Se

Huma es
ng, Estate e

ran
on
n ResouTrans

g
&
ort
rces port &Property
g

po
& RecruLogist

Ce Engin itmenics

Call cation &

Banki
Tra

D
Admin Const eering t

g
ng

,T
istrat & FinanEngin ructio
es ion cial eeringn
& Office

up

iv
Servic

Hulu Langat
AccouServic
ra

Se
cial Suppo
Edu nting es

ing
& Finan rt
ng Admi

i
Banki nistra Engin

si
n re Const eering

Go er
Manution &

eS
ructio
Sales

Kot
ructio tectu

r
Const Archi factu Office n

pa
n& al Admi

pur City Ce
Huma ring, Supp

on
cts

u
Desig & Medic Produ nistra n ResouTrans ort

t
hcare tion
umer & Office rces port

Se
Healt

fac
Cons & Logis
& Recru
l& ology
g/P

c
Supp

m
tics
be

Retai itmen

Ch ban
i
& Techn

ng
ort
ce Media ation t

f
Accou
ScienArts &rannu

g
/Ban
Bank Trade nting

u f
g, ing s&
rtisin & Supe

ba s
& FinanEngin Servi
Accou

re
n
tin & O
Adve ance

aK
Insur l cial eerinces

Ma
Lega Servi g
Manu
Reta
factu il
nting ces
ring,& Cons
Infor Call

m
Tran umer Sales

k
mati Cent

n ion
sport Prod
on re
& Com & CustEngin& Logisucts
Se

Ban
omer eerin tics

cou
mun

a
Adm Hosp icatioAcco Servg

K
inistr italit n untin ice
y Tech g

ina
ation
Desi Engi& Tour nolog
& Offic
Man gn neer ism y

Ku
ufac Minin & Arch Accoe Supp ing
Klan

Ma

int
t
turin g, itect untin ort
g, Reso
nu

Ac nistra
Tran urceure g
Kajang

factu Hosp
spor
italit t s & Ener

Johor
&
y & Logis gy
rin Ac Engi Tour tics
g, coun neer

t
Hum ism

balu
ing
Tran
gy
Adman Scie
tin

a
or
inistReso nce

gsar
g

lai
sp ratiource &
ort
Tech
n &s &

olo
Recr nolo
Offic
& Lo

mi
t
Info Engi uitm gy
e Supp
neer ent

en chn
pp
rma

gy
Ma tion gis ing ort
nu Mar & Com
tic
ntre

keti Trad
fact s
m
ng mun es
Edu icati & Serv

uit Te
&

Ad
ur Com

lo
catio on Saleices

Su
ing, Info mun
Tech s
icati n & Trai

cr ion
rma nolo
tion ons
Tr Scie Acco ning gy

les g no
an Adm& Com

Re
nce unti
inist mun
sp
t
&
Techng

s
ratio
or
& nica ion
icati nolo
n & on
t&

e
Sale gy
Offic Tech
s

ch
Man Acco
Lo e nolo

Sa erin
ufac unti Sup

s
fic
t
turi Scie gi ng port gy
Hu stics
ce u a En
ng, nce
m Tran

ur mm unic
gin
&

e
an spo Tech

f
rt nolo
Re Acc &

ee Sales
so
ounLog gy

T
Info Man

e so tingistic

o
rma ufacMar

rin
O
s
ur tion

gin Re & C mm
turiketi
ce Adm & ng,ng

g
Com

n
&
s& inis Tran Com

En an
tratmun spo mun

Co
ion icat rt
Re

& io
& ion & icat

n cr AccounOfficeTec Log ions

istic
ui
tio & ia
Suphno

at
m
logys
Scie tm ting por
Hum Ret

Hu rma n
ail nce
en t

g ed
an &
Mar Res Con &
Tec t

io ic
Info Adm sum
our hno

in & M ke rma inis ces

et ts
er Sale logy
tion Pro s
tin Adm trat &

fo
at
Hum ion EngRec Leg duc
inis &

un
g
In ark ising, Arrvices
trat Call an
Com & Con ineeruit al ts
& Offi stru
ringment

Ad
ion CenRes mu
Acc ce ctio
Co & tre our nica ounSup n

tr
Offi & ces tion
vice m ce Cus & Saletingport

M ve &
tomRecTechnos

m m
Ser Sup

g m
rt Se er ts un por er ruit
Ser me logy

is
tom duc Acc t

in icat
Man vicent
Cus Pro Ret oun
Ad es tre & er

in Com
Humufa Mar ail ting
io
ctu ket & Tra
ad l Cen Consum logsy
is ns
s

an ring ing Con des

in
Tr Res , & sum

tr
hno Adm &
& ourTra Com er Ser

r
s

Cal
Tec vice

En
ail inis Hea cesnsp
& Ser mu Pro vice
tic

istic

Ret

at
e ial trat lthc & ort nica

Sa
duc
enc re are Rec& Log tion tss

Ad
ee &
ion
Sci Financ

Ac gin
ctu g ty & Eng & ruit isti s

m
n

io
& ininper Off ine Med me
g ctio hite
gis

le
Tra Hum Ma ice erin ical ntcs
kin stru & Arc& Pro
an rke

m
& Sup g

n
Ad
s
BanCon ign ion

co ee
ate Resting por
Des catEst our & t
Lo

Edu l

in on

in
&
cesCom

un rin
Rea
& mu Sal
Rec nic es

is
O

In
ruit atio
&

tin g
Ma me ns

g ti
Log

tr
ffic

fo
nuf nt
act Hu Ma
ns

at
t

ma rke

rm
uri

En rma
or

ng, n ting
Cal Res

e
Tra

io
l
Heour & Com
e atio

nsp Cen

ati
g

althces
re S un sp

Su
ort tre mu Sal

n
& car & Rec
& nic

on
Log Cus Enge & rui atioes
in

an

isti tom ine Me tm ns

pp

&
gy

Acc cs er erindic ent

erv ic

Ma
Ser g al

&
nuf
le unt

oun

fo
act Re vic
hit sto om Tr

O
g

rt &

or
ting
uri tail e

Co
ort gisti hnolo

Ad ng, &

ffic
ic

ver TraCo

t
nsu

In
tu er m

tisi nsp
,

Ac up
ng, me
& ing

ort
Art &r Pro
Sa cco

in

m
Adm

e
S

co
En s Log duc
gin
al &

un
Me isti ts
eer
es

S
un
dia cs
r

in
s

ing
tu

is
ec

ic
u C

Hu

tin
Ma n
spo

tr
ec m

ma
cs
A

ati

rke Re
Cal
at
ke fac

ort

tin sou
l Cen

g
po
io
er

nT

g rce
t

Ban n
on

& s Eng & rucSer

en

De
n

tre

Co &
Edu Fin
kin
sig
&

mm Re ine
&
g

itm
u

rt
cat Coanc re
Ad

Cu

&
&

unicru ng g
sto
O

ion nstial
Arc C

Te
s entr tin

Arc
an

catitm
m

me
ffic
ru

tio

hite vice

ion ent
Tra tionvic
eri inin
r
&

in
Ma

Ser
upp
Rec

ch

s
ctu
e
Inf
istr
ran
M

Acc
Lo

orm

Sup
Ma atio
e

Hum

no
& O nsp nica
ne

Ma Res

oun

es
nuf n
&

at

rke
an

ting
Returin
act & Com

po
Ca ar

nu
Adm
es

Hu

io

ting ces

lo
ail g, mu ign
Man
&

rt
& Tra nic & Arc
rc

our

&
&

m
s

gy
Con nspatio
Res n

Com Rec
vice

Ac
M

&
De ll C

ou

Des nic
an ctio
Hum st n

an

sumort n Tec
Ser

mu
&
fact

Off
u

itm upp
Con ig

En Su

er & Loghno re
eS
ort
g
ial

Sal ducisti y
ufa
ru

Re Des

Pro
co

atio
ruit
anc

,T

gi
ice
mm
l

es ts cs
Man
dica

Info

me

hite
ns
Fin

rism

ne
so
Me

Ad ng,

Man tion

nt
tin

rma
&

ctu
gi
Tou

un

er
ctu

log
ur
g

&

inis

uf

ufa &
kin

Ret ket , mu
Rec sum inin Med s

Mar ring
are

uri
Tra s & vice

ecru e S
&

pp
in
ia
Ban

ce

ctu Com
en

ail ing Tra nica Con ce

min Tra
Hos lthc

lity

ac

Adm
ts
& Art Ser

g
duc

& & nsp tion stru Sup

pita

or
Co

s & Ar
Hea

ig
g

tin
ri

tu

Con Com ort

inis
&

Offic
tion Pro

Hos

t
n
Ret Legcat sing des

trat

sum mu & hno n t

rin
ng,
rea er

pita
Edu erti Tra

&
Re
ra
Sa oun
ion ,

ion
istr nsp
ffic

Hu

er nicaLog log
Info

Ac tionMar& Com Edu

lity
g,

g
trati

Pro tion
En

&
rt & Con

cr
urin

ent

Offi
&

rma
man n

&

duc
istics
Spoail al

Tec ctio por

Adv

ch
Tr
Man
ati ng, T

ui
Tou
&

Sa
coketingmunica&catiCusontomnciaOffilceSerSuplepors
ort

ts
ati

istic
tm
rism

s
an

ite
Call Baninistrat

le

s
on

Adm

un& Comtion &TecTraer Service

y
Re

en
sp

ct

s
Cenking
o

Ac
ment

ran
&R

cruitm

ur
tre &
or

t
uf

so
Hu minis
ing

Supp

tinmunication
Information
ati

e
t&
c

ur
on

ion
les

Ad

Fina&
ma

&
actu

co
Info
ri

on &

sport & Log

En
ce

g slogy
ort
on

Con
rma Call

Lo erin
Ac

rces

spo

n Re tio

Off
rm

hno g
s&

gine
act

Manufac
min factu

stru

Sa
tion

inin vices
un

gist g
Ac

Mar Comtre

ctio
&
Admi

&
ring,

keti mun
Re

Reang & icat tom

Cen
ice

n
sour n & Of Acco
& Recruit

&O

tra

ics
Info

t
fo

ou

l Esta

tin
crui
co
Lo
rt &
Enginee tion & Office
s & Re

&

Comion er
unt
Hum istr

te mun
Cus
ces

Su
In

Res

rma

&
gis

tm
En
nu

Prop
un

g
nuf

nistra

Tran

TechServ
Call Centre &
Administration

icat nolo
trati
ns

Administration & Office Support

& Re e Su ting

en

erty
pp
ring
ices municatio

tion

ions gy
Mini
turing,

tics
ffice

t
Ma

gin
& Communicat

Acc

ice
an

ng,

tin
Log

ScieReso Cons
fic

ort
cru ppor
Marke munica

nce urce truct

ion cial & Com

& Com

spor
Acco

Mark

& Tech
Serv

tion

itm
ce
y ucts

g
eeri
Finan eting
Ad

ure

s&
eting

Hum ing,
Ma

Farm
turing, Tran
r Prod
cal

isti

nolo gy
un

en
o

ting
itect

oun

Ener
Consing & Mark

ications
& Medi

nolog

Accounting

an Anim
inis

& Comm

gy
sour

ion
inee

Transpor
ume
thca& Arch

t
Resources

Reso als
Sup

Realurces & Cons

Engin
Desi truct

& Tech

& Of

t
Human

& Lo
RetaLega re

ce Cons

& Com Techno

ng
Banking

cs
Estat & Recruervat
Acc
Heal gn

ring

unica
Bank

Human Resources & Recruitment

l

gy

Informa
Hospitality
Scienil &

Marketing
Technolo

e&
tions
mun

Huma

eerin
tion

ti
nistra

Prope
Call
untin
Marketing & Communications

fice

gist
Customer Service

tion &
an Re

munica logy

n Resou
Adm

itmen
Resources

Centre
Human

rty t
port
ication

& Office Support

ng
Com

g
Design
ounting

ion
Commun
& Finan

Call Centre
Administr
Eng

ion Techno
Sales

Information & Communication Technology

rces& Archit Servic

& Custom

ics
Commun

t & Log

Resource

Supp
Sale
Accounting

& Recruecture e
& Comm
ng &

Engineering
ture

tion

ication

tion
Manufacturing, Transport & Logistics

Retail
ation
Accoun

& Custome

Healthca

er
& Conserva
& Architec

tion &

s & Recruitm
Admi

Marketing & Communication Technology

itmen
Engineering

cial Servi

& Consum
Technolo

Design Constru
& Office

s
Hum

Educatio
Marketi

& Recruit
& Property
Construc Informa

Training

gy

Manufac

ort
& Tourism

t
re &
Banking
Human
& Technolo

s
Advertising, Arts & Media

r Service

g
Design

& Architec
Design & Architecture

Enginee
Animals

Support

gy

n & Training
unications

istics

Medical

er Product
Communications
Educatio tion

ent
Design
Consulting
Farming, n &

Insurance & Superannuation

Retail & Consumer Products

Sales
Estate

Acc

& Financial
Sales

ring
Education & Strategy

& Property

ction
ting
Science & Technology
Information & Training

ture
Science

ces
Sales

& Architecture
Science & Technology
Design & Architecture

logy

s
Education & Training
Trades & Services

ment
Real

Sales
& Strategy
Services
&
Consulting

Real Estate

def remove_outliers_iqr(df, column):

Q1 = df[column].quantile(0.10)
Q3 = df[column].quantile(0.85)

IQR = Q3 - Q1

lower_bound = Q1 - 1.5 * IQR

upper_bound = Q3 + 1.5 * IQR

df_outlier_free = df[(df[column] >= lower_bound) & (df[column] <= upper_bound)]

return df_outlier_free

jobs = remove_outliers_iqr(job, 'Salary')

#===========================================================================================================#

# Get top 20 job titles from both DataFrames

top_leagues_job = job['job_title'].value_counts().nlargest(20).index
top_leagues_jobs = jobs['job_title'].value_counts().nlargest(20).index

# Combine the top job titles

top_leagues = top_leagues_job.union(top_leagues_jobs)

# Plotting
[Link](figsize=(16, 6))
[Link](x='Salary', y='job_title', data=job[job['job_title'].isin(top_leagues)])
[Link]('Distribution of Salary by Top 20 job_title from job')
[Link]('Salary')
[Link]('job_title')
[Link]()

[Link](figsize=(16, 6))
[Link](x='Salary', y='job_title', data=jobs[jobs['job_title'].isin(top_leagues)])
[Link]('Distribution of Salary by Top 20 job_title from After Removing Outliers')
[Link]('Salary')
[Link]('job_title')
[Link]()

top_leagues = jobs['job_title'].value_counts().nlargest(15)

[Link](figsize=(20, 8))

colors = sns.cubehelix_palette(len(top_leagues), light=0.7, dark=0.2)

bar_plot = [Link](x=top_leagues.index, y=top_leagues.values, palette=colors)

for index, value in enumerate(top_leagues.values):

label = f"{value:,}"
[Link](index, value + 0.1, label, ha='center', va='bottom', fontsize=15, color='#A52A2A')

[Link]('Top 15 Leagues by Player Count')

[Link]('League')
[Link]('Player Count')
[Link](rotation=90)
plt.tight_layout()

[Link]()

tips_df = pd.read_csv("[Link]")

display(tips_df.head(2))
fig = [Link](tips_df,
x="sex",
y="total_bill",
color="smoker",
barmode="group",
facet_row="time",
facet_col="day",
category_orders={"day": ["Thur", "Fri", "Sat", "Sun"],
"time": ["Lunch", "Dinner"]})
[Link]()

total_bill tip sex smoker day time size

0 16.99 1.01 Female No Sun Dinner 2

1 10.34 1.66 Male No Sun Dinner 3

day=Thur day=Fri day=Sat day=Sun

smoker
800
No
Yes

time=Lunch
600
total_bill

400

200

800

time=Dinner
600
total_bill

400

200

0
Male Female Male Female Male Female Male Female

sex sex sex sex

fig = [Link](tips_df,
x="time",
y="total_bill",
points="all")
[Link]()
 50

40
total_bill

30

20

10

Dinner Lunch

time

fig = [Link](tips_df,
x="time",
y="total_bill",
points="outliers")
[Link]()

50

40
total_bill

30

20

10

Dinner Lunch

time

fig = [Link](tips_df,
x="day",
y="total_bill",
color="smoker" )
fig.update_traces(quartilemethod="linear")
[Link]()
 smoker
50
No
Yes

40
total_bill

30

20

10

Sun Sat Thur Fri

day

fig = [Link](tips_df,
x="time",
y="total_bill",
color="smoker",
notched=True,
hover_data=["day"] # add day column to hover data
)
[Link]()

smoker
50
No
Yes

40
total_bill

30

20

10

Dinner Lunch

time

[Link](figsize=(20, 10))

x = jobs['job_title'].head(20)
y = jobs['Salary'].head(20)

# Plot the scatter plot with country names and numbers on y-axis
marker_sizes = jobs['Salary']
for i, country in enumerate(x):
[Link](country, [Link][i], s=(marker_sizes.iloc[i])/20, label=country, alpha=0.7)
[Link](country, [Link][i], f'{[Link][i]:,.0f}', ha='center', va='bottom', rotation='vertical', fontsize=10
# Set y-axis to display numbers in billions
plt.ticklabel_format(style='plain', axis='y', useOffset=False, scilimits=(9, 9))

[Link]('Job Title')
[Link]('Salary')
[Link]('Scatter Plot Job Title for Salary')
[Link](rotation=90)
[Link](True)
plt.tight_layout()

[Link]()

jobs = pd.read_csv("jobstreet_all_job_dataset.csv")
jobs = [Link](5000)
jobs = [Link](columns=['job_id'], axis=1)
jobs = jobs.reset_index()
jobs = [Link](columns=['index'], axis=1)
display([Link])
display([Link](2))

from wordcloud import WordCloud

text = str(list(jobs['category'])).replace(',', '').replace('[', '').replace("'", '').replace(']', '')

wordcloud = WordCloud(background_color = 'white', width = 1600, height = 800, max_words = 121).generate(text)

[Link](wordcloud)

[Link]('off')
[Link]()

(5000, 10)
job_title company descriptions location category subcategory role type salary listingDate

Mass JOB
Manager,
Rapid PURPOSE
Government Kuala 2024-05-
0 Transit :\nTo organize Construction Project Management manager Contract/Temp NaN
& Authority Lumpur 10T04:06:31Z
Corporation & participate in
Liasion
Sdn Bhd a ...

Designing
Saraya Information & information-
Junior IT solutions, Seremban 2024-04-
1 Goodmaid Communication Developers/Programmers technology- Full time NaN
Executive implementation, District 08T00:14:09Z
Sdn Bhd Technology executive
customiza...
# Drop rows with missing values and plot the resulting DataFrame
job = [Link]()

import re

def clean_and_calculate_mean(salary):
try:
# Remove currency symbols, words, and extra characters
salary = [Link]('RM', '').replace('MYR', '').replace('$', '').replace('per month', '').replace('p.m.'

# Handle ranges with different separators

if '–' in salary:
salary_range = [Link]('–')
elif '-' in salary:
salary_range = [Link]('-')
elif '—' in salary:
salary_range = [Link]('—')
else:
salary_range = [salary]

# Convert values to integers, handling potential errors

salary_values = []
for value in salary_range:
try:
value = int(float([Link](',', '').strip()))
salary_values.append(value)
except ValueError:
pass # Ignore non-numeric values

# Calculate mean if at least two valid values are found

if len(salary_values) >= 2:
salary_mean = sum(salary_values) / len(salary_values)
return salary_mean
else:
return None

except Exception as e:
print(f"Error processing salary '{salary}': {e}")
return None

# Apply the function to the salary column

job['Salary'] = job['salary'].apply(clean_and_calculate_mean)
job = [Link]('salary',axis=1)
[Link](2)

job_title company descriptions location category subcategory role type listingDate Salary

CTOS Data Attend to all inbound Call Centre & Customer

Specialist, Kuala call-centre- Full 2024-04-
4 Systems Sdn and outbound calls/ Customer Service - Call 2750.0
Contact Centre Lumpur role time 19T02:57:09Z
Bhd emai... Service Centre

Multilingual | Skills and Kampung Call Centre & Customer customer-

Private Full 2024-04-
5 Customer Support Abilities:\nSkilled Malaysia Customer Service - Call support- 5500.0
Advertiser time 05T12:56:47Z
Specialist communicator.\n... Raya Service Centre specialist

dfp = job['role'].value_counts().head(10).sort_values(ascending = True).reset_index()

dfl = job['location'].value_counts().head(10).sort_values(ascending = True).reset_index()
dfc = job['company'].value_counts().head(10).sort_values(ascending = True).reset_index()

fig = [Link]()

fig.add_trace([Link](y = dfp['role'],
orientation='h',
name = 'Position',
marker = dict(color = 'LightCoral')))

fig.add_trace([Link](y = dfl['location'],
orientation='h',
name = 'Location',
marker = dict(color = 'CadetBlue')))

fig.add_trace([Link](y = dfc['company'],
orientation='h',
name = 'Company',
marker = dict(color = 'SteelBlue')))

fig.update_layout(
updatemenus=[
dict(
type = "buttons",
direction="left",
pad={"r": 10, "t": 10},
showactive=True,
x=0.16,
xanchor="left",
y=1.12,
yanchor="top",
font = dict(color = 'Indigo',size = 14),
 buttons=list([
dict(label="All",
method="update",
args=[ {"visible": [True, True, True]},
{'showlegend' : True}
]),
dict(label="Position",
method="update",
args=[ {"visible": [True, False, False]},
{'showlegend' : True}
]),
dict(label='Location',
method="update",
args=[ {"visible": [False, True, False]},
{'showlegend' : True}
]),
dict(label='Company',
method="update",
args=[ {"visible": [False, False, True]},
{'showlegend' : True}]),
]),
)])

fig.update_layout(
annotations=[
dict(text="Choose:", showarrow=False,
x=0, y=1.075, yref="paper", align="right",
font=dict(size=16,color = 'DarkSlateBlue'))])

fig.update_layout(title ="Top 10 Positions, Locations and Companies",

title_x = 0.5,
title_font = dict(size = 20, color = 'MidnightBlue'))

[Link]()

Top 10 Positions, Locations and Companies

All
Choose: Position Location Company

Michael Page International (Malaysia) Sdn Bhd Position

Location
Ambition Group Malaysia Sdn Bhd
Company
Agensi Pekerjaan Hays (Malaysia) Sdn Bhd
MumsMe Sdn Bhd
Elabram Systems Sdn Bhd
Petaling
Selangor
Penang Island
Seberang Perai
Kuala Lumpur City Centre
sales-executive
accounts-executive
human-resource-executive
finance-executive
purchasing-executive
0 5

# Read data
df = pd.read_csv('US_Job_Market.csv')
[Link](3)

position company description reviews location

Development Director\nALS Therapy Development Atlanta, GA

0 Development Director ALS TDI NaN
... 30301

An Ostentatiously-Excitable Principal The Hexagon

1 Job Description\n\n"The road that leads to acc... NaN Atlanta, GA
Research... Lavish

2 Data Scientist Xpert Staffing Growing company located in the Atlanta, GA are... NaN Atlanta, GA

#!pip install dash

import pandas as pd
from dash import Dash, dcc, html
from [Link] import Input, Output
import plotly.graph_objects as go

dfd1 = df[df['position']== 'Data Scientist']

dfd2 = df[df['position']== 'Senior Data Scientist']
dfd3 = df[df['position']== 'Research Analyst']
dfd4 = df[df['position']== 'Data Engineer']

# Add 'position' column to each dataframe

redf1 = dfd1[["location", "position"]].value_counts().nlargest(10).sort_values(ascending = True).reset_index()
redf2 = dfd2[["location", "position"]].value_counts().nlargest(10).sort_values(ascending = True).reset_index()
redf3 = dfd3[["location", "position"]].value_counts().nlargest(10).sort_values(ascending = True).reset_index()
redf4 = dfd4[["location", "position"]].value_counts().nlargest(10).sort_values(ascending = True).reset_index()
# Create Plotly figure
fig = [Link]()

fig.add_trace([Link](x = redf1["location"],
y = redf1["count"],
marker = dict(color = 'Tomato'),
name = 'Data Scientist'))
fig.add_trace([Link](x = redf2['location'],
y = redf2['count'],
name = 'Senior Data Scientist',
marker = dict(color = 'LightCoral')))
fig.add_trace([Link](x = redf3['location'],
y = redf3['count'],
name = 'Research Analyst',
marker = dict(color = 'SteelBlue')))
fig.add_trace([Link](x = redf4['location'],
y = redf4['count'],
name = 'Data Engineer',
marker = dict(color = 'CadetBlue')))

# Update Layout with dropdown functionality

fig.update_layout(
updatemenus=[
dict(
direction="down",
pad={"r": 10, "t": 10},
showactive=True,
x=0.13,
xanchor="left",
y=1.12,
yanchor="top",
font = dict(color = 'Indigo',size = 14),
buttons=list([
dict(label="All",
method="update",
args=[ {"visible": [True, True, True, True]},
{'showlegend' : True}
]),
dict(label="Data Scientist",
method="update",
args=[ {"visible": [True, False, False, False]},
{'showlegend' : True}
]),
dict(label='Senior Data Scientist',
method="update",
args=[ {"visible": [False, True, False, False]},
{'showlegend' : True}
]),
dict(label='Research Analyst',
method="update",
args=[ {"visible": [False, False, True, False]},
{'showlegend' : True}
]),
dict(label='Data Engineer',
method="update",
args=[ {"visible": [False, False, False, True]},
{'showlegend' : True}]),
]),
)])

fig.update_layout(
annotations=[
dict(text="Choose:", showarrow=False,
x=0, y=1.075, yref="paper", align="right",
font=dict(size=16,color = 'DarkSlateBlue'))])

fig.update_layout(title ="The distribution of states by four Positions",

title_x = 0.5,
title_font = dict(size = 20, color = 'MidnightBlue'))

[Link]()
 The distribution of states by four Positions

80 Choose: All
▼
Data Scientist
Senior Data Scientist
Research Analyst
60 Data Engineer

40

20

0
Austin, TX

San Diego, CA

Seattle, WA

Atlanta, GA

Los Angeles, CA

Washington, DC

Chicago, IL

Boston, MA

San Francisco, CA

New York, NY

Mountain View, CA

Alameda, CA

New York, NY 10176

Washington, DC 20036

Sunnyvale, CA

San Mateo, CA
df = pd.read_csv("heart_disease_uci.csv")
df = [Link]()
[Link](2)

id age sex dataset cp trestbps chol fbs restecg thalch exang oldpeak slope ca thal num

lv fixed
0 1 63 Male Cleveland typical angina 145.0 233.0 True 150.0 False 2.3 downsloping 0.0 0
hypertrophy defect

lv
1 2 67 Male Cleveland asymptomatic 160.0 286.0 False 108.0 True 1.5 flat 3.0 normal 2
hypertrophy

g = [Link](x="chol", y="thalch", data=df, kind="kde", color="b")

g.plot_joint([Link], c="b", s=30, linewidth=1, marker="+")
g.ax_joint.collections[0].set_alpha(0)
g.set_axis_labels("chol", "thalch");

# --- Create Jointplot ---

# --- Create Jointplot ---
jointplot = [Link](x = 'age', y = 'chol', data = df, hue = 'sex', palette = 'PuRd')

# --- Jointplot Titles & Text ---

[Link]('Jointplot between Age and Chol', fontweight = 'heavy', y = 1.05, fontsize = '14',
fontfamily = 'sans-serif', color = 'black');

import [Link] as plt

# Define the labels

labels = ['chol', 'age', 'thalch','oldpeak']

# Calculate counts
counts = [df[label].sum() for label in labels]

# Create the bar plot

[Link](figsize=(10, 6))
bars = [Link](labels, counts, color=['skyblue', 'Red', 'Green'])

# Add value labels on top of each bar with some vertical offset
for bar, count in zip(bars, counts):
yval = bar.get_height() + 0.1 # Add a small offset
[Link](bar.get_x() + bar.get_width() / 2, yval, str(count), ha='center')

[Link]('Sum of Each Class')

[Link]('Class')
[Link]('Total')
[Link]()
[Link]['[Link]'] = (15,5)
[Link](1, 2, 1)
chart = [Link]('cp')['age'].mean().sort_values(ascending = False).plot(kind = 'bar', color = 'orangered')
chart.set_xticklabels(chart.get_xticklabels(), rotation = 0)
[Link]('Chest Pain Based on Age', fontsize = 15, color = 'b', pad = 12)
[Link]('Chest Pain')
[Link]('Age')

[Link](1, 2, 2)
chart = [Link]('thal')['oldpeak'].mean().sort_values(ascending = False).plot(kind = 'bar', color = 'gold')
chart.set_xticklabels(chart.get_xticklabels(), rotation = 0)
[Link]('Thal from Old Peak', fontsize = 15, color = 'b', pad = 12)
[Link]('Thal')
[Link]('Old Peak')
[Link]()

[Link](figsize = (12,4))
ax = [Link](x=[Link])
for bars in [Link]:
ax.bar_label(bars)
[Link]("Count of Levels", fontsize = 15);
[Link](figsize = (8,5))
[Link]([Link], shade = True, color = "r")
[Link]("Age Histogram", fontsize = 20)
[Link]()
print("Histogram's skewness is {} and kurtosis is {}".format([Link](), [Link]()))

Histogram's skewness is -0.21485314045391055 and kurtosis is -0.5174882052116159

import [Link] as stats

df_numeric = df.select_dtypes(include='number')

results = []

for col in df_numeric.columns:

skewness = df_numeric[col].skew()
kurtosis = df_numeric[col].kurt()
[Link]([col, skewness, kurtosis])

df_stats = [Link](results, columns=['Column', 'Skewness', 'Kurtosis'])

df_stats
 Column Skewness Kurtosis

0 id 0.90 3.95

1 age -0.21 -0.52

2 trestbps 0.70 0.80

3 chol 1.03 4.35

4 thalch -0.53 -0.09

5 oldpeak 1.24 1.52

6 ca 1.19 0.26

7 num 1.05 -0.16

from [Link] import norm

dfx = df[['chol', 'age', 'thalch','oldpeak']]

for col in dfx:

[Link](dfx[col],plot=plt)
[Link](col)
[Link]();
[Link](rc={'[Link]':(20,7)})
sns.set_style("white")
[Link](data=df, x="chol", y="trestbps", size="oldpeak", hue='cp',legend=True, sizes=(10, 500));

[Link](rc={'[Link]':(20,7)})
[Link](y='trestbps',x='chol',data=df,kind='scatter',size='oldpeak',hue='cp',aspect=1.2);

# Filter out rows with 'oldpeak' equal to 0.0

df_filtered = df[df['oldpeak'] != 0.0]

# Create the countplot

[Link](figsize=(20, 7))
[Link](data=df_filtered, x='oldpeak', order=sorted(df_filtered['oldpeak'].unique()))

# Access bars through the current axes

for bar in [Link]().patches:
[Link](bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.1, int(bar.get_height()), ha='center', va='botto

# Add labels and title

[Link]('Tumor Size Distribution (Excluding 0.0)')
[Link]('Number of Patients')
[Link]('Oldpeak Value')
[Link](rotation=0) # Rotate x-axis labels for better readability
[Link]()

[Link](figsize=(20, 7))
# Create the countplot
[Link](data=df, x='age', order=sorted(df['age'].unique()))

# Access bars through the current axes

for bar in [Link]().patches:
[Link](bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.1, int(bar.get_height()), ha='center', va='botto

# Add labels and title

[Link]('Age Distribution of Patients')
[Link]('Number of Patients')
[Link]('Age')
[Link]()

import [Link] as plt

import seaborn as sns
from scipy import stats # Import stats module

def diagnostic_plots(df, variable):

[Link](figsize=(17, 5))
[Link](1, 3, 1)
[Link](df[variable])
[Link]('Histogram')

[Link](1, 3, 2)
[Link](df[variable], dist="norm", plot=plt) # Use [Link]
[Link]('RM quantiles')

[Link](1, 3, 3)
[Link](x=df[variable])
[Link]('Boxplot')
 [Link]()

for col in df[['age','chol']].select_dtypes(exclude="O").columns[:20].to_list():

diagnostic_plots(df,col)

corr = df.select_dtypes('number').drop('id',axis=1).corr()
# Generate a mask for the upper triangle
mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True
fig, ax = [Link](figsize=(15,10))
[Link](corr, cmap='Spectral_r', mask=mask, square=True, annot=True, linewidth=0.5, cbar_kws={"shrink" : 0.5
df = [Link]('id',axis=1)
[Link](2)

age sex dataset cp trestbps chol fbs restecg thalch exang oldpeak slope ca thal num

0 63 Male Cleveland typical angina 145.0 233.0 True lv hypertrophy 150.0 False 2.3 downsloping 0.0 fixed defect 0

1 67 Male Cleveland asymptomatic 160.0 286.0 False lv hypertrophy 108.0 True 1.5 flat 3.0 normal 2

[Link](figsize=(20, 5))
sns.set_context("paper")

kdeplt = [Link](
data=df,
x="chol",
hue="sex",
palette='Dark2',
alpha=0.7,
lw=2,
)

kdeplt.set_title("Cholesterol values distribution\nMale VS Female", fontsize=12)

kdeplt.set_xlabel("Cholesterol", fontsize=12)

# Calculate mean cholesterol for each sex

mean_male = df[df['sex'] == 'Male']['chol'].mean()
mean_female = df[df['sex'] == 'Female']['chol'].mean()

# Add vertical lines for mean cholesterol

[Link](x=mean_male, color="#2986cc", ls="--", lw=1.3)
[Link](x=mean_female, color="#c90076", ls="--", lw=1.3)

# Add text annotations

[Link](mean_male, [Link]().get_ylim()[1], f"Mean Cholesterol / Male: {mean_male:.2f}",
fontsize=10, color="#2986cc", ha='right', va='top')
[Link](mean_female, [Link]().get_ylim()[1], f"Mean Cholesterol / Female: {mean_female:.2f}",
fontsize=10, color="#c90076", ha='left', va='top')

[Link]()
heart_df_fg = [Link](
data=df,
col="sex",
hue="sex",
row="cp",
height=4,
aspect=1.3,
palette='Dark2',
col_order=["Male", "Female"],
)
heart_df_fg.map_dataframe([Link], "age", "chol")
[Link]()
x = [Link]("cp")["chol"].min().index
y = [Link]("cp")["chol"].min().values
df = [Link]({'cp':x,
'chol':y })

fig = [Link](df,
x='cp',
y='chol',
color='cp', #color represents brand
title='Chol Value'
)
[Link]()

Chol Value

180
cp
asymptomatic
160 atypical angina
non-anginal
140 typical angina

120

100
chol

80

60

40

20

0
asymptomatic atypical angina non-anginal typical angina

cp

df = pd.read_csv("heart_disease_uci.csv")
df = [Link]()
df = [Link]('id',axis =1 )
[Link](2)

age sex dataset cp trestbps chol fbs restecg thalch exang oldpeak slope ca thal num

0 63 Male Cleveland typical angina 145.0 233.0 True lv hypertrophy 150.0 False 2.3 downsloping 0.0 fixed defect 0

1 67 Male Cleveland asymptomatic 160.0 286.0 False lv hypertrophy 108.0 True 1.5 flat 3.0 normal 2

[Link](figsize=(18,5))

[Link](1,5,1)
[Link](df['age'],color='DeepPink')
[Link](1,5,2)
[Link](df['chol'],color='Green')
[Link](1,5,3)
[Link](df['thalch'],color='Red')
[Link](1,5,4)
[Link](df['oldpeak'],color='Magenta')

plt.tight_layout()
[Link]()
df_cpy = [Link]("Deep")
df_cpy = df_cpy.select_dtypes("number")
df_cpy = df_cpy[['age','chol','thalch','oldpeak']]

fig,axis=[Link](ncols=4,nrows=1,figsize=(15,5))
index=0
axis=[Link]()

for col,values in df_cpy.items():

[Link](y=col,data=df_cpy,color='r',ax=axis[index])
index+=1
plt.tight_layout(pad=0.5,w_pad=0.7,h_pad=5.0)

df_cpy = [Link]("Deep")
df_cpy = df_cpy.select_dtypes("number")
df_cpy = df_cpy[['age','chol','thalch','oldpeak']]

flierprops = dict(markerfacecolor='g', color='g', alpha=0.5)

n_cols = 4
n_rows = int([Link](df_cpy.shape[-1]*2 / n_cols))
fig, axes = [Link](n_rows, n_cols, figsize=(4 * n_cols, 3 * n_rows))
for i, (col) in enumerate(list(df_cpy.columns)):
mean = df_cpy[col].mean()
median = df_cpy[col].median()
[Link](df_cpy[col], ax=[Link]()[2*i], kde=True)
[Link](x=df_cpy[col], orient='h', ax=[Link]()[2*i+1], color='g')
[Link]()[2*i+1].vlines(mean, ymin = -1, ymax = 1, color='r', label=f"For [{col}]\nMean: {mean:.2}\nMedian:
[Link]()[2*i+1].legend()

if i % n_cols == 0:
ax.set_ylabel('Frequency')
else:
ax.set_ylabel('')
plt.tight_layout()

[Link](style='whitegrid', palette="deep", font_scale=1.1, rc={"[Link]": [20, 6]})

[Link](df['chol'], bins = 30).set(xlabel = "Chol");

df2 = df[['age','sex','chol']]

f, (ax_box, ax_hist) = [Link](2, sharex=True, gridspec_kw={"height_ratios": (.15, .85)})

ax_box.title.set_text('Age countplot and Boxplot')

[Link](df2["age"], orient="h" ,ax=ax_box)
[Link](data=df2, x="age", ax=ax_hist)
ax_box.set(xlabel='')
[Link]()

#is online delivery available?

colors = ("darkorange", "green",'Red','Pink')
explodes = [0.5, 0.5,0.75, .50]
df["cp"].value_counts(sort=False).[Link](colors=colors,
textprops={'fontsize': 15},
autopct = '%4.1f',
startangle= 90,
radius =2,
rotatelabels=True,
shadow = True) ;
wine = pd.read_csv("[Link]")
[Link](2)

fixed volatile citric residual free sulfur total sulfur

chlorides density pH sulphates alcohol quality Id
acidity acidity acid sugar dioxide dioxide

0 7.4 0.70 0.0 1.9 0.08 11.0 34.0 1.0 3.51 0.56 9.4 5 0

1 7.8 0.88 0.0 2.6 0.10 25.0 67.0 1.0 3.20 0.68 9.8 5 1

NUMERICAL = wine[['fixed acidity', 'volatile acidity', 'residual sugar',

'chlorides', 'free sulfur dioxide', 'total sulfur dioxide',
'pH', 'alcohol']]
fig, axes = [Link](2, 4)
fig.set_figheight(12)
fig.set_figwidth(16)
for i,col in enumerate(NUMERICAL):
[Link](wine[col],ax=axes[(i // 4) -1 ,(i % 4)], kde = True)
axes[(i // 4) -1 ,(i % 4)].axvline(wine[col].mean(), color='k', linestyle='dashed', linewidth=1)
#set configuration for charts
[Link]["[Link]"]=[18 , 6]
[Link]["[Link]"]=15
[Link]["[Link]"]="medium"
[Link]["[Link]"]="medium"

def plot_disribution(data , x ,color,bins ):

mean = data[x].mean()
std = data[x].std()
info=dict(data = data , x = x , color = color)
[Link](1 , 3 , 1 , title =f"Ditstribution of {x} column")
[Link](a=data[x] , bins = bins)
[Link](f"bins of {x}")
[Link](mean , label ="mean" , color ="red")
[Link]("frequency")
[Link](["${\sigma}$ = %d"%std , f"mean = {mean:.2f}"])
[Link](f"histogram of {x} column")
[Link](1 , 3 , 2)
[Link](**info)
[Link](f"{x}")
[Link](f"box plot of {x} column")
[Link](1 , 3 , 3)
[Link](**info)
[Link](f"{x}")
[Link](f"distribution of points in {x} column")
[Link](f"Distribution of {x} column" , fontsize =15 , color="red")
[Link]()

age_bins = [Link](29 , 77+5 , 5)

base_color = sns.color_palette()[4]
plot_disribution(data = df , x ="chol" , color = base_color , bins=age_bins)
plot , ax = [Link](1 , 3 , figsize=(20,6))
[Link](data = [Link][df["thal"]== 'normal'] , x = "age" , hue = "sex",binwidth=2,ax = ax[0],palette = sns.
[Link](data = [Link][df["thal"]== 'reversable defect'] , x = "age" , hue = "sex",binwidth=2,ax = ax[1],palette
[Link](data = [Link][df["thal"]== 'fixed defect'] , x = "age" , hue = "sex",binwidth=2,ax = ax[2],palette
[Link]()

sex = ["Male", "Female"]

values = df["sex"].value_counts()
color = ["#FF0000", "#000000"]

[Link](figsize = (5, 7))

[Link](values, labels = sex, colors = color, explode = (0.1, 0), textprops = {"color":"w"}, autopct = "%.2f%%",

[Link]();

#plotting
fig, (ax1, ax2) = [Link](1, 2, figsize=(18, 9))
[Link](' Highest and Lowest Correlation ', size = 20, weight='bold')
axs = [ax1, ax2]
#kdeplot
[Link](data=df, y='chol', x='thalch', ax=ax1, color="red")
ax1.set_title('Chol Vs Thalch', size = 14, weight='bold', pad=20)

#kdeplot
[Link](data=df, y='chol', x='oldpeak', ax=ax2, color='Blue')
ax2.set_title('Chol Vs Oldpeak', size = 14, weight='bold', pad=20);

df1 = pd.read_csv('US_Job_Market.csv')
df1 = [Link]().reset_index()
df1 = [Link]('index',axis=1)
[Link](2)

position company description reviews location

Operation DEPARTMENT: Program OperationsPOSITION Atlanta, GA

0 Data Analyst 44.0
HOPE LOCATIO... 30303

Assistant Professor -TT - Signal Processing & Emory

1 DESCRIPTION\nThe Emory University Department o... 550.0 Atlanta, GA
... University

[Link](figsize=(20, 7))

# Filter for the top 10 most frequent companies

df_v = df1['company'].value_counts().head(10).reset_index()
df_v.columns = ['company', 'count']

# Calculate the percentage

total = df1['company'].value_counts().sum()
df_v['percentage'] = (df_v['count'] / total) * 100

# Create the bar plot

plot = [Link](y='company', x='count', data=df_v)

# Annotate the bars with the percentage

for index, row in df_v.iterrows():
[Link](row['count'], index, f"{row['percentage']:.2f}%", color='black', ha="left")

[Link](rotation=0)
[Link]('Top 10 Most Frequent Companies')
[Link]()
from PIL import Image
import [Link] as plt
import numpy as np
from wordcloud import WordCloud, STOPWORDS
import pandas as pd
import requests
from io import BytesIO

# Download mask image

mask_url = "[Link]
response = [Link](mask_url)
mask_image = [Link](BytesIO([Link]))
wordcloud_mask = [Link](mask_image)

# Generate word cloud

[Link](figsize=(15,15))
all_text = " ".join(df1['company'].[Link]())
wordcloud = WordCloud(width=800,
height=800,
stopwords=STOPWORDS,
background_color='white',
max_words=800,
colormap="hsv",
mask=wordcloud_mask).generate(all_text)

# Display the word cloud

[Link](wordcloud, interpolation='bilinear')
[Link]('off')
[Link]()

from wordcloud import WordCloud

# create a word cloud for positive reviews

positive_reviews = df1[df1['location'] == 'Atlanta, GA']['company'].[Link](sep=' ')
positive_cloud = WordCloud(width=1500, height=800, max_words=100, background_color='white').generate(positive_reviews

[Link](figsize=(20, 6), facecolor=None)

[Link](positive_cloud)
[Link]("off")
plt.tight_layout(pad=0)
[Link]()

import plotly.graph_objs as go
values = df1['company'].value_counts()[:10]
labels=[Link]
text=[Link]
fig = [Link](data=[[Link](values=values,labels=labels,hole=.3)])
fig.update_traces(hoverinfo='label+percent', textinfo='value', textfont_size=20,
marker=dict(line=dict(color='#000000', width=3)))
fig.update_layout(title="Most popular Jobs in USA",
titlefont={'size': 30},
)
[Link]()

Most popular Jobs in USA

[Link]
Ball Aerospace
Microsoft
Google
187 NYU Langone Health
Fred Hutchinson Cancer Research Center
357
KPMG

137 Broad Institute

Facebook
Walmart eCommerce

134
45
49

76
70 66 49

print("Count of unique Jobs in USA")

locationCount=df1['company'].value_counts().head(10).sort_values(ascending=True)
locationCount
fig=[Link](figsize=(18,10))
[Link](kind="barh",fontsize=8)
[Link]("Job names",fontsize=25,color="red",fontweight='bold')
[Link]("Jobs Vs. COUNT GRAPH",fontsize=40,color="BLACK",fontweight='bold')
for v in range(len(locationCount)):
[Link](v+locationCount[v],v,locationCount[v],fontsize=10,color="BLACK",fontweight='bold')

Count of unique Jobs in USA

z=df1['position'].value_counts().head(10)
fig=[Link](z,x=[Link],y=[Link],color=[Link],text=[Link],labels={'index':'job title','y':'count','text':'count'
[Link]()

Top 10 Popular Roles in Data Sceince

200
position
204
Data Scientist
Senior Data Scientist
Research Analyst
150 Data Engineer
Machine Learning Engineer
Sr. Data Scientist
count

Principal Data Scientist

100
Quantitative Analyst
Research Scientist
Lead Data Scientist
50
53
44
39
26 22 20 20 20 17
0
Da Se R Da Ma Sr Pr Qu Re Le
ta nio ese ta ch . in an se ad
Sc rD ar
c En ine Data cipa tit ar
c D
ien ata hA gin L S lD ati h S ata
tis na ee ea cie ata ve cie Sc
t Sc lys r rn n A n ien
ien ing tis Sc n
tis t En t ien alys tist tis
t
t gin tis t
ee t
r

position

# Plotting Outliers
col = 1
[Link](figsize = (20, 10))
for i in [Link]:
if col < 11:
[Link](2, 5, col)
[Link](wine[i])
[Link](i)
col = col + 1
s = [Link](x = 'cp',data = df)
sizes=[]
for p in [Link]:
height = p.get_height()
[Link](height)
[Link](p.get_x()+p.get_width()/2.,
height + 3,
'{:1.2f}%'.format(height/len(df)*100),
ha="center", fontsize=16)

#checking the target variables for distribution

[Link](df['chol'],color='Red')
[Link](x=df['chol'].mean(), color='Blue', linestyle='--', linewidth=2)
[Link]('Chol');

[Link][:, :-1].describe().T.sort_values(by='std' , ascending = False)\

.style.background_gradient(cmap='GnBu')\
 .bar(subset=["max"], color='#BB0000')\
.bar(subset=["mean",], color='green')

count mean std min 25% 50% 75% max

total sulfur dioxide 1143.000000 45.914698 32.782130 6.000000 21.000000 37.000000 61.000000 289.000000

free sulfur dioxide 1143.000000 15.615486 10.250486 1.000000 7.000000 13.000000 21.000000 68.000000

fixed acidity 1143.000000 8.311111 1.747595 4.600000 7.100000 7.900000 9.100000 15.900000

residual sugar 1143.000000 2.532152 1.355917 0.900000 1.900000 2.200000 2.600000 15.500000

alcohol 1143.000000 10.442111 1.082196 8.400000 9.500000 10.200000 11.100000 14.900000

quality 1143.000000 5.657043 0.805824 3.000000 5.000000 6.000000 6.000000 8.000000

citric acid 1143.000000 0.268364 0.196686 0.000000 0.090000 0.250000 0.420000 1.000000

volatile acidity 1143.000000 0.531339 0.179633 0.120000 0.392500 0.520000 0.640000 1.580000

sulphates 1143.000000 0.657708 0.170399 0.330000 0.550000 0.620000 0.730000 2.000000

pH 1143.000000 3.311015 0.156664 2.740000 3.205000 3.310000 3.400000 4.010000

chlorides 1143.000000 0.086933 0.047267 0.012000 0.070000 0.079000 0.090000 0.611000

density 1143.000000 0.996730 0.001925 0.990070 0.995570 0.996680 0.997845 1.003690

df[df["age"] >= 50].describe().style.background_gradient(cmap='RdPu')

age trestbps chol thalch oldpeak ca num

count 213.000000 213.000000 213.000000 213.000000 213.000000 213.000000 213.000000

mean 59.159624 134.793427 252.220657 144.708920 1.214085 0.835681 1.103286

std 5.731645 18.575307 54.953695 22.377966 1.197004 0.969460 1.280713

min 50.000000 94.000000 100.000000 71.000000 0.000000 0.000000 0.000000

25% 55.000000 120.000000 214.000000 130.000000 0.100000 0.000000 0.000000

50% 58.000000 132.000000 246.000000 148.000000 1.000000 1.000000 1.000000

75% 63.000000 145.000000 283.000000 161.000000 1.800000 1.000000 2.000000

max 77.000000 200.000000 564.000000 195.000000 6.200000 3.000000 4.000000

def highlight_min(s, props=''):

return [Link](s == [Link]([Link]), props, '')
[Link]().[Link](highlight_min, props='color:yellow;background-color:Grey', axis=0)

age trestbps chol thalch oldpeak ca num

count 299.000000 299.000000 299.000000 299.000000 299.000000 299.000000 299.000000

mean 54.521739 131.715719 246.785953 149.327759 1.058528 0.672241 0.946488

std 9.030264 17.747751 52.532582 23.121062 1.162769 0.937438 1.230409

min 29.000000 94.000000 100.000000 71.000000 0.000000 0.000000 0.000000

25% 48.000000 120.000000 211.000000 132.500000 0.000000 0.000000 0.000000

50% 56.000000 130.000000 242.000000 152.000000 0.800000 0.000000 0.000000

75% 61.000000 140.000000 275.500000 165.500000 1.600000 1.000000 2.000000

max 77.000000 200.000000 564.000000 202.000000 6.200000 3.000000 4.000000

Prepared By: Syed Afroz Ali