09/03/2026, 22:23 OneNote

pandas
29 January 2026 09:39

Data types supported :

1) int
2) float
3) string/object
4) bool
5) datetime
6) category

Data access, data filtering:

Column:
[Link][0]--access by location I.e index
[Link][:, 2]]--access by location I.e index
df['col_name']--access by name/ value (this is stored in series)
df[['col1','col2']]-- this stores in dataframe, as 2 cols need 2 dimn hence can't be in series
Row:
[Link](3)--- accessing top 3 values
df[1]--by index
df[0:2]--for range
df[-3:-1]--for last 3 values (-1 shows last value)
[Link][:3]--gives based on index (I.e 0,1,2)
[Link][:3]--gives based on row_id(0,1,2,3)

Row and col :

[Link][3,'Order_ID']--specific value
df.[[0,4]]--both oth and 4th
df.[[0,3],['c1']]
df.[:,'c1']
[Link][[0,2],2:3]
***loc: when using col name and row index to access data(both the start and stop labels are inclusive)
Iloc: int based access, row index and col index (start is inclusive and the stop is exclusive)
Row doesn't usually have labels other than in index

Naming unlabelled columns:

df2 = pd.read_csv(filename1, header=None)--this is explicitly mention that the column names are missing , this will be like a place holder and puts 0...n bef
row that is the data will be overwritten)
[Link] = ["DBAName", "Facility Type", "Risk","Address",'Zip','InspectionDate','InspectionType', 'Results', 'Violations']--renaming the columns
df2 = df2.set_index("Reg_No")--this is to specify that instead of default row index use the column reg_no as new row index
df2 = df2.reset_index(drop=False)--this is to turn to default , but if you don’t mention drop=false ,it will delete the existing row index values
[Link]=['courses']--naming the column/label
[Link](columns={'DURATION':'TIME'},inplace=True)---renaming one specific column

Dealing with null values :

[Link]().sum()-- int
[Link]().any()-- bool
[Link](0)-- filling the NA values of any col with specified values(in this case 0)
[Link](df['TIME'].mean())-- filling with the mean of a specific col
df['TIME'].fillna(df['TIME'].mean(), inplace=True)
[Link]()-- dropping the rows that have all the cols as null

Duplicates:
[Link]()-- dropping the rows that have all the cols as null
[Link]().any()--searching the whole df to see if any dupe cols are present
[Link](subset=['MOVIE_TITLE']).any()-- this to chcek if any specific col has dupes
df_drop_dup = df.drop_duplicates('MOVIE_TITLE')-- to drop the specific col dupe row

Conditional column data extraction:

df2[df2['Status'] == "failure"]
df1[df1['Facility Type'].isna()]
df1[(df1['Violations'].isna()) & (df1['Results'] == 'Pass')]
df1[df1['InspectionDate'].[Link]('2017')]

[Link] 1/4
09/03/2026, 22:23 OneNote
Data inspection : 1) structural :size, shape, null , not null, column, dupe ([Link]() is actually for structural but can be used for uni) **outlier
2 ) uni-variate : considering one variable/aspect([Link]())
3) bi-variate
On integers:
1) aggregate values / stats values:
df['Fund Sort ascending'].min()
df['Fund Sort ascending'].max()
df['Fund Sort ascending'].sum()
df['Fund Sort ascending'].mean()
df['Fund Sort ascending'].median()
df['Fund Sort ascending'].mode()
df['Fund Sort ascending'].std()

Data inspection vs data analysis: DI: knowing the facts/just viewing or auditing them ([Link] etc)
Da : observing the pattern , or finding the cause( like frequency , distribution)

Converting lists, dict into dataframe:

List:
course=['vlsi','mvt','cys','cc','cs']--list
df=[Link](course)--conversion
[Link]=['courses']--naming the col
Dict:
data = {
"student_id": [101, 102, 103, 104, 105, 106],
"attendance": [92, 85, 98, 70, 88, 76],
"internal_marks": [40, 35, 45, 30, 38, 28],
}--dict
df_scores = [Link](data)
Int this the key field would be taken as col names

Data extraction:
• loc,iloc
Boolean--df1[df1["Marks"] != 0]
• Query()--[Link]('Marks != 0')
• Where()--[Link](df1['Marks'] == 0)---gives NAN when the condition fails
[Link](df1['Marks'] == 0)---can customise the msg for the nan o/p
• Filter()
• df[df["program"].isin(["Cybersecurity","AI"])]-- using isin() to filter out the data
• df['program'].value_counts()-- to find the frequency of a field
• Nunique()--works on whole df and individual col, but unique()--works when col mentioned

Data visualization :
• df['program'].value_counts().plot(kind='bar')--creating bar graph using pandas, can use line, hist ,box,pii,area(line filled w color),barh(horizontal),pie
• Matplotlib, seaborn – library
• Box plot : for numeric values, not for strings , gives summary of the data (like avg value of the data ), also uni-variat inspection, to know distribution
▪ For the summary such as : max, min, mean , median , outliers
▪ More the data ,better the plot
▪ If the difference/variation among the data is more..then avoid this
▪ Components : min, max ,median , quartile Q1(25%), quartile Q3(75%), IQR=Q3-Q1, lower bound = Q1-1.5*IQR , upper bound= Q3+1.5*IQR
▪ df[['rating','shelf']].plot(kind='box')
▪ [Link](column =['carbo','sugars'])
▪ figsize=(10,5) It's used to specify the size of the plot area in inches. The first value (10) represents the width of the figure, and the second value (5) re
▪ [Link](kind='line', subplots=True, figsize=(15, 8))
▪ Depending on the distribution the plot can be left skewed or right skewed
▪ Violin plot , notched box plot(one more parameter notch=true/false), boxen plot, swarm plot – variants in the box plot
• Pie chart :categorization (like male, female)
▪ Difficult if the values are less or more than 5 categories ( use waffle if more )
▪ Donut is a kind of pie chart , nested pie chart , rose chart
▪ import [Link] as plt
▪ df['mfr'].value_counts().plot(kind='pie')
▪ df['shelf'].value_counts().plot(kind='pie',autopct='%.1f%%')
▪ [Link]([Link], labels = [Link], autopct='%1.2f%%')-- lats parameter to specify the numeric values on the pie chart
▪ [Link]() -- to show the chart
▪ [Link](labels, loc='lower right',title="course name:")-- it is like the guide box, to specify the colour to label mapping (purple-cys, blue-aiml)
▪ [Link](sizes, labels = labels, autopct='%1.2f%%',colors=['violet','lightgreen','tomato','r'])-- to use colours
▪ Elevated/ explode chart
○ [Link](sizes, labels = labels, autopct='%1.2f%%',colors=['violet','lightgreen','tomato','r'],explode=(0.2,0.1,0,0))-- explode parameter shows the eleveat
○ [Link](sizes, labels = labels, autopct='%.2f%%',colors=['violet','lightgreen','tomato','m'],explode=(0.2,0,0,0), shadow=True, startangle=90)--start angle

▪ Donut chart :
○ d = [Link](sizes, labels=labels, autopct='%.1f%%',
startangle=90, labeldistance=0.95)

[Link] 2/4
09/03/2026, 22:23 OneNote
○ [Link]().add_artist(circle)-- this is to create the inner circle
• Bar chart : univariate, but want to compare multiple values
▪ fuel_type_counts = df['fueltype'].value_counts()-- count values for each of the category
▪ import [Link] as plt
▪ fuel_type_counts.plot(kind='bar')-- to plot graph
▪ [Link]('Distribution of Fuel Types')--graph title
▪ [Link]('Fuel Type')-- labelling x axis values
▪ [Link]('Number of Cars')--labelling y axis
▪ Barh—for horizontal bar graph
a. Bi-variat :
• Scatterplot: co-relation btwn 2 aspects (increasing +ve co relation , decreasing –ve co relation)
▪ [Link](kind='scatter', x='sugars', y='shelf')--have to specify on which axis we need which value
• Bubble chart – for 3 variable

Feature Bar Graph Histogram

Data Type Qualitative/Categorical Quantitative/Continuous
Spacing Bars have gaps between them. Bars are touching (no gaps).
Order Can be rearranged (e.g., alphabetical). Must follow a numerical order (intervals).
Width of Bars Usually uniform; width doesn't matter. Represents the "bin" or interval size.
Example Sales by different car brands. Number of people in different age groups.

Data sorting :
• df.sort_values("program")
• df.sort_values("program").head(2) -- first 2 values
• df.sort_values("program", ascending=[False])-- descending
• df.sort_values(["program","marks"], ascending=[True, False])-- first it'll sort the program in asc then amongst each alpha it sorts marks desc
• df.sort_values(["marks","program"], ascending=[False, True])-- this is as same as sorting the marks, hence the above one works fine

String methods :
• df['Gender']=df['Gender'].[Link]()
• df['Gender']=df['Gender'].[Link]()-- removes the leading or trailing spaces
• df['SALARY']=pd.to_numeric(df['SALARY'].[Link]('$', ''), errors='coerce')--string replacement
• df['FULL_NAME'] = df['FIRST_NAME'] + ' ' + df['LAST_NAME']-- concat string
• df['EMAIL_DOMAIN'] = df['EMAIL'].apply(lambda email: [Link]('@')[1] if '@' in email else None)--splitting the string (splittint at @)

ML :
“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as m
experience E.”
1) supervised learning
• Labelled data set : the data with the input and the labelled output(like marks and results as p/f)
• Both I/p and o/p is know
▪ Classification : separable kind of data , op is category
○ Decision tree, random forest
▪ Regression : predicting value -- cvss scoring , op is continuous value

2) unsupervised learning
• Unlabelled data set :A dataset where only input data is given — no correct answers.
• Model must discover patterns itself
▪ Clustering
▪ Dimensionality/feature reduction
▪ Association rule learning
3) reinforced learning
• Learning by trial and error.
• Reward based

--

[Link] 3/4
09/03/2026, 22:23 OneNote

[Link] 4/4