PYTHON

1. What is variable?
Variable is name that refers to a value stored in memory. It is used to store data and access it later.
Declaring data type separately not necessary.
Example: x=10 # integer data type
name=”Alice” #string data type
price=19.99 #float data type
value=True #boolean datatype

2. What is identifiers?
It is name used to identify variables, functions, class, objects.
Rules for Identification/defining variable:
 Can include letter(a-z or A-Z), digits(1-9), underscore(_)
 Cannot start with a digit or any other symbol except underscore(_)
 Are case sensitive
 Example: _my_name=”Ritesh”
my_no=91
Both are different variable because it is case-sensitive.
My_no=91

3. What is Pre-defined keywords?

Keywords are reserved words in python that have special meaning and specific purpose. They cannot
used as variable.
Example: if,else,class,import,def,elif,break,continue,class,etc.

4. What is Data type?

In Python, a data type classifies the kind of value a variable holds. Python is dynamically typed, which
means the data type of a variable is determined at runtime based on the assigned value, without
explicit declarations. Understanding data types is crucial for writing efficient and error-free code.

5. How to identify the datatype?

We can identify datatype using type() built in function.
Example: a=2
type(a) # integer data type

b=”ritesh”
type(b) #string data type

6. Type Conversion: Convert one data type into another

There are two types of casting:
 Implicit No need to declare any datatype, Python itself declare any it based on the data passed
into the variable.
 Explicit
 Example: a=”2”
type(a) #string type
int(a) #2
type(a) #integer type

Note: to convert string into integer there should be integer value inside string.
We cannot convert string (which contains float value) into integer it will throw error.

7. How to input from user?

Input from user will always result into string
Example: a=input(“Enter your name:”)
print(a)
Output: Enter your name: Ritesh
‘Ritesh’

8. How Memory for string is allocated in system?

Example: name=”HELLO PYTHON”

how to access “T”

name[8] #output: T
name[-] #output T
 Operators
1. Arithmetic operator

Operator Meaning Example

+ Addition 3+2=5
- Subtraction 5-2=3
* Multiplication 3*2=6
/ Division 6 / 2 = 3.0
// Floor Division 7 // 2 = 3
% Modulus (remainder) 7%2=1
** Exponentiation 2 ** 3 = 8

2. Comparison Operator: Compare two values and returns Boolean values

Operator Meaning Example

== Equal to 3 == 3 → True

!= Not equal to 3 != 4 → True

> Greater than 5 > 2 → True

< Less than 3 < 5 → True

>= Greater or equal 4 >= 4 → True

<= Less or equal 3 <= 5 → True

Note: when we compare string then in reality ascii value of first letter of each is compared
Example: ‘Ajay’>’Sanjay’ #output False because ascii value of ‘S’ is greater than ‘A’

3. Logical Operator: Used with Boolean values(True,False)

Operator Meaning Example

and Logical AND True and False → False

or Logical OR True or False → True

not Logical NOT not True → False

Note: using bool() function we can check value is True or False

Example: bool(1) #output True
bool(5) #output True
bool(“ritesh”) #output: True
bool(0) #output: False
bool(None) #output: False

4. Assignment Operator: assigning values to variable

Operator Meaning Example

= Assign x=5

+= Add and assign x += 2

-= Subtract and assign x -= 2

*= Multiply and assign x *= 3

/= Divide and assign x /= 2

//= Floor divide and assign x //= 2

%= Modulus and assign x %= 3

**= Power and assign x **= 2

5. Membership operator: Check whether any member is present or not(“in” and “not in”)

Operator Meaning Example

in Exists in 'a' in 'apple' → True

not in Does not exist 'x' not in 'apple' → True

6. Identity operator: It is used to compare memory location of two objects. (“is”, “is not”)
Example:
 Example 1: a=5
b=3
a is b #output: False
This means that a has different memory location and b has different memory location. Also
they pointing to different value.
To check memory location we can use id() function
 Example 2: a=2 #id(a)-> 1432
b=a #id(b)-> 1432
In this example memory location of two variables is same and they pointing to one same
value.

Operator Meaning Example

is Same object a is b (True if same)

is not Not same object a is not b

7. Bitwise Operator: Operations at bit level, they perform operations on individual bits (0 and 1)
numbers.

Operator Meaning Example

& AND 5&3=1

` ` OR
Operator Meaning Example

^ XOR 5^3=6

~ NOT ~5 = -6

<< Left Shift 2 << 1 = 4

>> Right Shift 4 >> 1 = 2

8. Order of precedence in python

Precedence Operators Description Example

1 () Parentheses (2 + 3) * 4

2 ** Exponentiation 2 ** 3 = 8

Unary plus, minus,

3 +x, -x, ~x -5, ~5
bitwise NOT

Multiplication,
4 *, /, //, % 10 / 2
Division, Modulo

5 +, - Addition, Subtraction 5+3

6 <<, >> Bitwise shift 4 << 1 = 8

7 & Bitwise AND 5&3=1

8 ^ Bitwise XOR 5^3=6

9 ` ` Bitwise OR

==, !=, >, <, >=, <=, is, is a == b, a in

10 Comparisons
not, in, not in b

11 not Logical NOT not True

True and
12 and Logical AND
False

True or
13 or Logical OR
False

Conditional expression x if a > b

14 if – else
(ternary) else y

15 =, +=, -=, *=, /=, etc. Assignment x=5

lambda
Lambda expression lambda x: x
16
+1
 Flow Control
1. Conditional Statements are used to make decisions and run different code based on whether a
condition True or False
(a) if: condition is True then block of code will be executed
Example: a=100
if a>0:
 print(“The number is greater than 0”)
output: The number is greater than 0

(b) else: if statement condition if False, then else block of statement code will be executed.
Example: weather=”rainy”
if weather==”sunny”:
print(“I will play cricket”)
else:
print(“I will not play cricket”)
output: I will not play cricket

(c) elif: elif statement condition is True then block of code will be executed.
Example: age=18
if age>18:
print(“You are bigger”)
elif age==18:
print(“You are just 18”)
else:
print(“You are smaller”)
output: You are 18

(d) nested if-else: You can have multiple if-else inside if-else
Example:
x=1
y=6
if x<5:
if y>5:
print(“x is less than and y is greater than 5”)
else:
print(“Both are less than 5”)
else:
print(“x is greater than 5”)
output: x is less than and y is greater than 5

2. Loop Statement: It allows you to execute a block of repeatedly

(a) while loop: This statement repeatedly execute a block of code until a condition is met.
Example: n=5
i=1
while i<n:
print(i,end=”,”)
i+=1

output: 1,2,3,4

(b) break statement: Terminate/exit the loop

Example: i=1
while i<5:
print(i,end=”,“)
i+=1
 if i==3:
break

output: 1,2

(c) continue statement: It skips the particular iteration

Example: i=1
while i<6:
i=i+1
if i==3”
continue
print(i,end=”,“)

output 2,4,5

(d) for loop: Iterate a sequence of elements or iterables(list,string,tuples,dictionaries)

Example: for i in “pwskills”:
print(i,end=”, “)

output: p, w, s, k, i, l, l, s

Example: list=[1,2,”Ritesh”,True,12.4]
for i in list:
print(i)
else:
print(“This will be executed when for loop ends successfully without a break statement”)

output: 1
2
“Ritesh”
True
12.4
This will be executed when for loop ends successfully without a break statement

(e) range(): It print sequence of numbers

syntax: range(start,stop,step)
example: list(range(0,5)) #list having range function, we can pass range is any data iterables
such as tuples and many more.

Output: [0,1,2,3,4]

Example: for i in range(0,5):

print(i,end=”, “)

output: 0,1,2,3,4
Note: The stop value won’t be printing.

Data Structure
 1. what is Data Structure?
It is a way of organizing and storing data so that it can be accessed and manipulate efficiently.

2. Mutability:
 An object is mutable if its content (value) can be changed after creation.
 List, Set, Dictionaries can be mutable.
 Operation can be done: add, remove, modify, change
 If this operation is performed then no new object or memory location is created but instead
changes are done on the same object.

3. Immutability:
 An object is immutable if its content cannot be changed after creation.
 int, float, bool, string, tuple can be immutable.
 Operation cannot be done: add, remove, modify
 If we reassign value then new object or memory location will be created or error will be shown
when try to modify, changes done creates new object.
 Example 1: #int
x=5
print(“value of x is”,x,”and id is”,id(x)) #value of x is 5 and id is 14322
x=10 #reasssigning value or even tried to change eg: x+=5 then to new memory location and
object will be created
print(“value of x is”,x,”and id is”,id(x)) #value of x is 10 and id is 14052

4. List: It can store anything.

 It can store heterogenous data (such as string, Boolean, float, None).
 List is an ordered collection of elements that can be of any data type.
 List are mutable.
 Values stored inside square brackets i.e. [ ]
 Values can access by index number.
 We can concatenate two lists
 Example: num=[1,2,”Ritesh”,True,12.4]
type(num) #List
num[2] #’Ritesh’
num[0] #1
for i in num:
print(i)

output: 1
2
‘Ritesh’
True
12.4

5. Tuples: It can store any data type.

 Tuples are immutable.
 It is ordered collection of elements.
 Values can be accessed by index number.
 Values Stores inside parenthesis i.e. ()
 Example: tup= (1, 2, ”Ajay”, True, None, 13.4)
tup[2] # ‘Ajay’
6. Sets: It can store any data type but elements should be unique means no repeatition.
 It is unordered collection of elements.
 It is mutable.
 Values store inside curly braces i.e. { }
 Values can only be accessed using for loop
 Example: s={1, 2, “ajay”, None}
for i in s:
print(i,end=”,”)

output:
1, 2, ‘ajay’, None

7. Dictonaries: It can store data in key & value pair separated by colon.
 It is mutable and ordered collection of elements.
 Value can only be accessed using key pair.
 Value is stored inside Curly braces.
 Key pair must store only mutable data type (string, int, float, Boolean) and no special character like @, $,
%, etc.
 Value pair can store both mutable & immutable data type (list, string, tuples, sets, int, float, etc.)
 Example: dic = {‘name’: ‘ajay’, ‘age’: 22}
dic[‘name’] # ‘ajay’
 Object Oriented Programming System(OOPS)
Example: Suppose Ajay and Vijay want admission in IIT. IIT gives them zerox copy of original
form(template). Form contains details like name, age, address, marks, etc. So here original form is
class and zerox copy are objects of the class. While details (name, age, etc..) are attributes/properties
of form. Here original form also known as template/blueprint.
1. Class
The class is a user-defined data structure that combines data members and methods into a single
entity. For the object creation, classes serve as a blueprint/template. You can make as many objects
from class because objects are unique.
Syntax: class ClassName:
#attributes and methods.
NOTE: ClassName should be in UpperCamel case. UpperCamel case means first letter of each word
should capital and words should not contain space. Example: MyClass(), BankDetails(), etc.

Example:
Class Car:
color=”red”
def drive(self):
print(“The car is driving”)
ob1=Car() #object creation
[Link] # ‘red’
[Link]

here, when object is created then class Car is called then it must be followed by parenthesis while
calling. Without parenthesis class won’t be called.

2. Object: An instance of a class is called object. It consists of sets of methods and attributes. Action is
done using objects. We can create many objects. Objects store data and each object has different
value. Each name of object must be different.
Syntax: object_name=ClassName()

Example:
class Info():
name=”ritesh” #class attribute
def value(self):
print(“My name is”,[Link])

ob1=Info()
[Link] #’ritesh’
[Link]() #My name is ritesh

3. Attribute: It is variable that belongs to the class or object of that class. It holds data.
There are two main types of attributes in Python:

 Instance Attributes
 These attributes are specific to each object or instance of a class. They are defined inside the
__init__ method (constructor) using the self keyword. Each instance of the class will have its
own copy of instance attributes, and their values can be different for each instance.
 Class Attributes
These attributes are shared by all instances of a class. They are defined directly within the class
definition, outside of any methods. Class attributes are accessed using the class name itself or
self. Changes made to a class attribute through the class name will affect all instances.

Example:
class Bank:
name=”ritesh” #class variable
def __init__(self,balance):
[Link]=balance #instance variable
def display(self):
print(f”Mr. {[Link]} has balance left: {[Link]}”)

ob1=Bank(1000) #values is passed while creating object of the instance variable(inside

constructor)
[Link] #’ritesh’
[Link] #1000
[Link]() #Mr. ritesh has balance left: 1000

ob2=Bank(1500)
[Link] #’ritesh’
[Link] #1500
[Link]() #Mr. ritesh has balance left: 1500

Here, object is unique and each object has different value stored jaise balance gets changes
whenever new objects get created.

Example: Of class attribute affects all instance of the class

class Student:
count=0 #class attribute
def __init__(self,name):
[Link]=name
[Link]+=1 #accessing class attribute using class(Student) we can also access
#using cls inside @classmethod because cls refers to class(Student) itself

@classmethod
def update_count(cls,new_count):
[Link]=new_count
#creating instance
s1=Student(“Aay”)
s2=Student(“Bijay”)
[Link] #2 (Before modification) here, we can also access class attribute using
ClassName(Student)
 #Modifying class attribute using @classmethod
Student.update_count(5)
[Link] #5
@classmethod(update_count) modified class variable and this effect
All instance of the class. Means when @classmethod(update_count)
is called, count value changes. Count value remains same even called
using any object. This shows how update_count(classmethod) effects
to all instance of the class.
[Link] #5

4. Function inside class is known as method. There are three types of method in python.
Instance Methods:
 These are the most common type of methods.
 They operate on a specific instance of the class and can access and modify its data.
 The first parameter of an instance method is always self, which refers to the instance itself.
Class Methods:
 These methods are bound to the class and not the instance of the class.
 They can access and modify class-level data.
 They are defined using the @classmethod decorator and take cls as the first parameter, which
refers to the class itself.
 classmethod is an alternative of __init__(self) method. Actually classmethod is overloading
the __init__(self) method

Static Methods:
 These methods are not bound to either the class or an instance.
 They are defined using the @staticmethod decorator and do not take self or cls as
parameters.
 They are often used for utility functions related to the class.
 The staticmethod can also be called without creating instance of the class

Example:
class MyClass:
def __init__(self, attribute):
[Link] = attribute

def instance_method(self):
print(f"This is an instance method. Attribute: {[Link]}")

@classmethod
def class_method(cls):
print(f"This is a class method. Class: {cls}")

@staticmethod
def static_method():
print("This is a static method.")

#object creation
obj = MyClass("value")
obj.instance_method() # Output: This is an instance method. Attribute: value
 MyClass.class_method() # Output: This is a class method. Class: <class
'__main__.MyClass'>
MyClass.static_method() # Output: This is a static method.

Example of clasmethod
class Student:
count=0 #class attribute
def __init__(self,name):
[Link]=name
[Link]+=1 #accessing class attribute using class(Student) we can also access
#using cls inside @classmethod because cls refers to class(Student) itself

@classmethod
def update_count(self,new_count):
[Link]=new_count
#creating instance
s1=Student(“Aay”)
s2=Student(“Bijay”)
[Link] #2 (Before modification) here, we can also access class attribute using
ClassName(Student)

#Modifying class attribute using @classmethod

Student.update_count(5)
[Link] #5
@classmethod(update_count) modified class variable and this effect
All instance of the class. Means when @classmethod(update_count)
is called, count value changes. Count value remains same even called
using any object. This shows how update_count(classmethod) effects
to all instance of the class.
[Link] #5

Example of static method:

class Calculator:
@staticmethod
def @add(a,b);
return a+b
Calculator(2,3) #5
 Example of classmethod overloading the init method:
class Student:
def __init__(self,name):
[Link]=name
@classmethod
def student_details(cls,name1):
return cls(name1)

ob1=Student.student_details("Ajay")
[Link] #’Ajay’

Here, you will get ‘Ajay’ inspite of new name1 because it is overloading the
__init__(self) method.

5. If we want to execute method then use self as first parameter in method, when method
is defined. If we do not use self then method won’t execute.
Self: Self is a naming convention used within class definitions. It is a variable that allows you to
access attributes and methods within class. While self is a commonly used, you can replace it with
any valid name. The purpose of self is to ensure that each object recognize that method is associated
with it. In other words self acts as a reference to the specific instance of the class. When multiple
objects are created from same class, ‘self’ helps maintain separation between their data. Without
‘self’, python would not know which object’s data is being referenced. It ensures that correct
instance attribute and methods are accessed allowing individual objects to function independently.

Example: without self, throws error

Class Car:
def drive(): # no self is passed as parameter in method definition
print(“This is car”)

c1=Car()
[Link]() #throws error

Example: with self

class Car():
def drive(self):
print(“This is car”)

c1=Car()
[Link]() #This is car

Example:
class Personal:
def __init__(self,name,age):
[Link]=name
[Link]=age
def display(self):
print(f”My name is {[Link]} and age is {[Link]}”)
 ob1=Personal(“ritesh”,21)
[Link]() #My name is ritesh and age is 21
[Link] #’ritesh’
[Link] #21

6. __init__(): It is dunder/magic method. It is also a constructor. This method means initialization of

variable. While making object of the class, the first method is executed is __init__ and this method
just needs an argument while calling ClassName during object creation. This allows to take attribute
value differently for each object.
Example:
class ListOps:
def __init__(self,lst):
[Link]=lst
def even(self):
lst=[Link]
lst2=[]
for i in lst:
if i%2==0:
[Link](i) #even number
return lst2
ob1=ListOps([1,2,3,4,5,6,7,8])
[Link]() #[2,4,6,8]

Inheritance
1. Inheritance refers to the process where a child/sub/derived class receives/access the properties
and methods of parent/super/base class.
Syntax:
class BaseClass: # parent/super/base class
#body of base class
class DerivedClass: # child/sub/derived class
#body of derived class

2. Types of inheritance: single inheritance, multi-level inheritance, multiple inheritance,

hierarchical inheritance, hybrid inheritance.
3. Single inheritance: when a child class has only one parent class.
Example:
class Father:
def father_property(self):
print(“This is fathers class”)
class Child(Father):
def child_property(self):
print(“This is child class”)
 child_obj =Child()
child_obj.chld_property() #This is child class
child_obj.father_property # This is father class

#here, child can access both property fathers and his own class. But father can’t access child
class property, he can only access his class property

father_obj=Father()
father_obj.father_class() #This is father class

4. Multi-level inheritance:
 File Handling, Exception Handling, Logging and Debugging,
Multiprocessing and Multithreading

Multiprocessing
1. Multiple process run at different process that is known as multiprocessing. Multiple process run at
the same time parallel. Process don’t share memory between each other.
2. Sequential Execution (One Function at a Time)
Example:
import time

# Start the timer

start = time.perf_counter()

# Function that prints messages and sleeps for 1 second

def test_func():
print("Doing something...")
print("Sleeping for 1 second...")
[Link](1)
print("Done sleeping!")

# Run the function twice sequentially

test_func()
test_func()

# Stop the timer

end = time.perf_counter()

# Print how long it took

print(f"The program finished in {round(end-start, 2)} seconds")

Output:
do something
sleep for 1 sec
done with sleeping
do something
sleep for 1 sec
done with sleeping
 The program finished in 2.0 seconds

Explanation:
 The program executes test_func() two times in order.
 Since each function call pauses for 1 second, the total time taken is about 2 seconds.

3. Using Multiprocessing for Parallel Execution

Example: Running Two Processes

import multiprocessing
import time

# Start the timer

start = time.perf_counter()

def test_func():
print("Doing something...")
print("Sleeping for 1 second...")
[Link](1)
print("Done sleeping!")

# Create two separate processes

p1 = [Link](target=test_func)
p2 = [Link](target=test_func)

# Start both processes

[Link]()
[Link]()

# Wait for both processes to finish

[Link]()
[Link]()

# Stop the timer

end = time.perf_counter()

print(f"The program finished in {round(end-start, 2)} seconds")

output:
do something
sleep for 1 secdo something

sleep for 1 sec

done with sleeping
done with sleeping
The program finished in 1.05 seconds

Explanation:
 Instead of running the function one after another, two separate processes run at the same time.
 Since both sleep simultaneously, the total time taken is around 1 second instead of 2.

4. Example: Running 10 Processes in Parallel

import multiprocessing
import time

start = time.perf_counter()

def test_func():
print("Doing something...")
print("Sleeping for 1 second...")
[Link](1)
print("Done sleeping!")

# Create 10 processes
processes = []
for i in range(10):
p = [Link](target=test_func)
[Link]() # Start each process
[Link](p)

# Wait for all processes to complete

for process in processes:
[Link]()

end = time.perf_counter()
print(f"The program finished in {round(end-start, 2)} seconds")

Explanation:
 Normally, running the function 10 times would take 10 seconds.
 Using multiprocessing, all 10 processes start at the same time.
 The total execution time is only around 1 second.

5. Multiprocessing for Computation

Example: Squaring Numbers Using Multiple Processes

import multiprocessing
import time

start = time.perf_counter()

# Function to square a number

def square(index, value):
value[index] = value[index] ** 2
 # Shared array between processes
arr = [Link]('i', [1, 2, 5, 3, 40000000000])

# Create multiple processes to square numbers

processes = []
for i in range(len(arr)):
p = [Link](target=square, args=(i, arr))
[Link]()
[Link](p)

for process in processes:

[Link]()

print(list(arr)) # Print the squared numbers

end = time.perf_counter()
print(f"The program finished in {round(end-start, 2)} seconds")

Output:
[1, 4, 25, 9, 822083584]
The program finished in 0.05 seconds

Explanation:
 Instead of squaring each number one by one, processes compute simultaneously.
 Multiprocessing works well for CPU-intensive tasks.

6. Using Multiprocessing Pool for Simpler Code

import multiprocessing
import time

start = time.perf_counter()

# Function to square a number

def square(no):
result = no * no
print(f"The square of {no} is {result}")

numbers = [1, 2, 3, 4, 6000]

# Using a pool of processes for parallel execution

with [Link]() as pool:
[Link](square, numbers)

end = time.perf_counter()
print(f"The program finished in {round(end-start, 2)} seconds")
 Output:
The square of 1 is 1 .
The square of 3 is 9 .The square of 2 is 4 .

The square of 4 is 16 .The square of 6000 is 36000000 .

The program finished in 0.1 seconds

Explanation:
 [Link]() handles multiple processes without manual start and join.
 Results may not be in order, showing processes ran in parallel.

7. Using Multiprocessing with Queues (Real-Life Example)

Imagine a school where students need to enroll and register:
One process adds enrollment requests to a queue.
Another process processes those requests.
Example:

import multiprocessing

# Function to add enrollment requests

def enroll_students(student_queue):
for student in ["Rahul", "Rohit", "Aman", "Ajay"]:
student_queue.put(f"Enrollment request for {student}")

# Function to process the registration requests

def register_students(student_queue):
while True:
enrollment_req = student_queue.get()
if enrollment_req is None:
break
print(f"Registering the enrollment request: {enrollment_req}")

# Create a queue for communication between processes

student_queue = [Link]()

# Create processes for enrolling and registering students

enrollment_process = [Link](target=enroll_students, args=(student_queue,))
registration_process = [Link](target=register_students, args=(student_queue,))

# Start both processes

enrollment_process.start()
registration_process.start()

# Wait for both to finish

enrollment_process.join()
registration_process.join()
 output:

Explanation:
 While one process adds requests, the other processes them.
 This allows tasks to run efficiently in parallel
Why are we passing student_queue as an argument?
 When creating enrollment_process and registration_process, we are passing student_queue as an
argument to both functions (enroll_students and register_students).
 Reason: We need both functions to communicate with the same queue. The enrollment process
puts student requests into the queue, and the registration process retrieves and processes those
requests.
What is [Link]()?
 [Link]() is a special queue that enables processes to share data safely.
 It is useful when multiple processes need to exchange information.

8. Using [Link] for Simpler Multiprocessing

import [Link]
import time

start = time.perf_counter()

def test_func(i):
print(f"Task {i}: Doing something...")
print("Sleeping for 1 sec...")
[Link](1)
print("Done sleeping!")

# Using ProcessPoolExecutor for cleaner syntax

with [Link]() as executor:
[Link](test_func, range(10))

end = time.perf_counter()
print(f"The program finished in {round(end-start, 2)} seconds")

output: The program finished in 0.73 seconds

Instead of managing processes manually, ProcessPoolExecutor handles it efficiently

 Data Toolkit
1. What is CRISP Dm Framework?
The CRISP-DM framework stands for Cross-Industry Standard Process for Data Mining. It is a
methodology used to guide data mining and data science projects, ensuring they are structured,
repeatable, and effective.
CRISP-DM Meaning:
Cross-Industry: Designed to work across different industries (finance, retail, healthcare, etc.).
Standard Process: A well-defined set of steps that standardizes how data mining is done.
Data Mining: Extracting useful patterns, trends, or knowledge from large sets of data.

CRISP-DM Framework Phases:

 Business Understanding
o Focuses on understanding the project objectives and requirements from a business
perspective.
o Defines the business problem and goals clearly.
 Data Understanding
o Involves collecting initial data and familiarizing with it.
o Identifies data quality problems, discovers initial insights, and detects interesting subsets.
 Data Preparation
o Covers all activities needed to construct the final dataset from the raw data.
o Tasks include cleaning, transforming, and selecting relevant data.
 Modeling
o Applies various modeling techniques and calibrates model parameters.
o Often requires back-and-forth with data preparation.
 Evaluation
o Assesses the model for quality and effectiveness.
o Determines if the model meets business objectives.
 Deployment
o Involves deploying the model into production.
o Can include generating reports, implementing systems, or delivering insights to stakeholders
2. Why python libraries are essential for data understanding to data preparation?
Python libraries like NumPy and Pandas are essential for data understanding and preparation because
they provide powerful, efficient, and easy-to-use tools for handling large datasets. NumPy offers fast
numerical operations and array manipulation, while Pandas simplifies data cleaning, transformation, and
exploration through dataframes. These libraries help quickly identify patterns, handle missing values,
and reshape data, which are crucial steps before applying any machine learning or analytical models.
Without them, processing and preparing data would be much slower and more complex.

3. What is library, package, module?

Library: Library is a collection of pre-written code that is used to perform common task. It is collection
of packages.
Package: Package is a directory containing many modules(programming files). It is collection of
modules.
Modules: Modules is a script(program) stored in a “.py” extension file. Modules contains functions
and classes that perform some specific task.

4. There are different libraries in python such as numpy, pandas, matplotlib, seaborn, etc…
 Numpy
Sure! Here's your content formatted more consistently and clearly:

1. Introduction to NumPy
 NumPy stands for "Numerical Python."
 Created in 2005 by Travis Oliphant to make mathematical computations faster in Python.
 Before NumPy, Python had different mathematical tools (similar to specialized software for
calculations).
 It provides support for large, multi-dimensional arrays and matrices, along with a collection of
high-level mathematical functions for efficient operations.
 It stores homogeneous data, meaning all elements in an array must be of the same data type (e.g.,
an array containing only integers).

2. Importing NumPy and Checking Its Version

Example:
import numpy as np # 'np' is an alias (short name) for NumPy
np.__version__ # Displays the version of NumPy installed
print(np.__doc__) # Shows a brief description about NumPy
Explanation:
 import numpy as np allows us to use np instead of writing numpy every time.
 np.__version__ lets us check which version of NumPy is currently installed.
 print(np.__doc__) provides a brief description of NumPy.
Note: If a library is not installed, install it using the command:
!pip install numpy # Example command to install NumPy

3. Lists vs. NumPy Arrays

Example (Python List):
lis = [1, 2, 3, "pwskills", 3+5j, True, 1.2] # Stores different types of data
type(lis) # Check type (It will be 'list')
Explanation:
 Lists can store heterogeneous (mixed) types of data, such as:
o Integers (1, 2, 3)
o Strings ("pwskills")
o Complex numbers (3+5j)
o Boolean (True)
o Floating-point numbers (1.2)
 Lists are stored in scattered memory locations, making access slower than arrays.
Example (NumPy Array):
l = [1, 2, 3, 4, 5]
arr = [Link](l) # Convert list to NumPy array
type(arr) # Check type (It will be '[Link]')
Explanation:
 NumPy arrays store homogeneous data (all elements must be of the same type).
 Arrays are faster than lists because:
o They store data in continuous memory locations, which speeds up computation.
o Since all elements are of the same type, calculations happen efficiently.

4. Creating NumPy Arrays with Different Data

Example:
l = [1, 2, 3, 4, 5, "Ajay"]
[Link](l) # Converts all elements to strings
Explanation:
  If the list contains mixed types (numbers and strings), NumPy converts everything to strings.
Example:
l = [1, 2, 3, 4, 5, 2.5]
[Link](l) # All elements become floating-point numbers
Explanation:
 If numbers and decimals exist, NumPy ensures they match the highest precision type (e.g., floats).

5. Dimensions, shape, size in Arrays

Example:
arr=[Link]([1,2,3,4])
[Link] # Get the number of dimensions (1D array)
Explanation:
 ndim tells us how many dimensions an array has.
Example:
arr1 = [Link]([[1, 2, 3], [4, 5, 6]])
[Link] # This is a 2D array
Explanation:
 If there is more than one pair of square brackets, it’s a multi-dimensional array.
Example:
arr2=[Link]([[[1,2],[3,4],[5,6]]])
[Link] #This is a 3D array
Explanation:
 Since there are three pairs of square brackets, this is a 3D array.
Example:
arr=[Link]([[1,2,3],[4,5,6]])
[Link] # (2,3)
 This tells the shape of the array, 2 rows and 3 columns
Example:
[Link] #6
 This tells the size of array, length of array

6. Creating Matrices
Example:
mat = [Link]([1, 2, 3, 4])
type(mat) # Check type (It will be '[Link]')
[Link] # Matrices are always 2D
Explanation:
 Matrices are specialized arrays with a fixed 2D structure.
 If we try to create a 3D matrix, NumPy will throw an error.

7. Creating Arrays in Other Ways

Example:
l = [1, 2, 3]
[Link](l) # Converts list to array
[Link](l) # Same as above, but keeps special array types like matrix
Explanation:
 [Link]() keeps special array types unchanged (e.g., matrices).
 Example:
mat = [Link]([1, 2, 3, 4]) # Matrix
[Link](mat) # Output: matrix([[1, 2, 3, 4]])

8. Copying Arrays (Shallow vs. Deep Copy)

  Shallow Copy: Changes in one copy affect the other.
Example (Shallow Copy):
a = arr # Both 'a' and 'arr' point to the same memory location
a[0] = 5000 # Changing 'a' also changes 'arr'

 Deep Copy: Changes in one copy do not affect the original array.
Example (Deep Copy):
b = [Link]() # Creates a separate copy
b[0] = 11000 # Changes in 'b' do NOT affect 'arr'

1. Using Functions to Generate Arrays

Example:
arr1 = [Link](lambda i, j: i == j, (3, 3)) # here (3,3) is setting the shape of the array
Output:
array([[ True, False, False],
[False, True, False],
[False, False, True]])
Explanation:
 [Link]() creates an array using a function.
 i == j compares row and column positions, returning True or False.
Example:
arr2 = [Link](lambda i, j: i * j, (3, 3))
Output:
array([[0., 0., 0.],
[0., 1., 2.],
[0., 2., 4.]])
Explanation:
 Multiplies row index (i) with column index (j).

2. Creating Arrays from Iterators

Example:
iterator = (i for i in range(5)) # Generator object
[Link](iterator, float) # Converts generator to a NumPy array
Output:
array([0., 1., 2., 3., 4.])

3. Converting Strings to NumPy Arrays

Example:
[Link]('22 23 24', sep=" ")
Output:
array([22., 23., 24.])
Explanation:
 Converts numerical strings into a NumPy array.
 By default, values are floats.
 This fromstring() function only accepts numerical value inside function
Example:
[Link]('22 23 24', sep=" ", dtype=int)
Output:
array([22, 23, 24])
Explanation:
 Specifying dtype=int ensures integer conversion.
4. Creating Number Sequences
Example:
[Link](1, 10) # Generates numbers from 1 to 9, it is similar to range function
Output:
array([1, 2, 3, 4, 5, 6, 7, 8, 9])
Example:
[Link](1, 5, 10) # Generates 10 evenly spaced numbers from 1 to 5
Output:
array([1. , 1.44, 1.89, 2.33, 2.78, 3.22, 3.67, 4.11, 4.56, 5. ])
Explanation:
 linspace() is useful for graphs and mathematical computations.
 Numpy Advance 1

1. Importing NumPy
Example:
import numpy as np
Explanation:
 This command imports the NumPy library, which helps us work with arrays and perform
mathematical operations easily.

2. Creating Arrays Filled with Zeros

Example:
a = [Link](5)
a #output: array([0., 0., 0., 0., 0.])
[Link] #output: 1
Explanation:
 [Link](5) creates an array with 5 elements, all set to zero.
 [Link] returns the number of dimensions of the array. Since a is a 1D array, the output will be 1.
 We can also mention data type as int eg. [Link](5,dtype=int) output:[0,0,0,0,0]
Example:
b = [Link]((3, 4))
b #output: array([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])

[Link] # 2
Explanation:
 [Link]((3, 4)) creates a 2D array (a table) with 3 rows and 4 columns, all filled with zeroes.
 [Link] returns the number of dimensions of the array, which is 2 (since it has rows and columns).

3. Creating Arrays Filled with Ones

Example:
[Link](5) #output: array([1., 1., 1., 1., 1.])
Explanation:
 Creates a 1D array with 5 elements, all set to 1.
Example:
[Link]((3, 4),dtype=int) #output: array([[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]])

Explanation:
 Creates a 2D array with 3 rows and 4 columns, all filled with ones.

4. Checking If [Link] Exists

Example:
[Link](5) #throws error
Explanation:
 This does not exist in NumPy. There's no built-in function to create an array filled with twos, but
you can use:
[Link](5, 2)
 [Link](5, 2) creates an array with 5 elements, all filled with 2.
5. Creating Arrays Manually
Example:
[Link]([[2, 2, 2], [2, 2, 2]])
Explanation:
 Defines a 2D array manually with two rows, each containing [2, 2, 2].
Example:
[Link]((3, 4)) #output: array([[4., 4., 4., 4.],
[4., 4., 4., 4.],
[4., 4., 4., 4.]])
Explanation:
 Creates a 3×4 array without initializing values (random junk data).
 Return a new array of given shape and type, without initializing entries.
 It means entries in the array can be anything random values

6. Creating Multi-Dimensional Arrays

Example:
arr = [Link]((3, 3, 4))
[Link]
Explanation:
 [Link]((1, 3, 4)) creates a 3D array with:
o 3 blocks
o Each block has 3 rows
o Each row has 4 columns
 Think of it as multiple 2D tables stacked together.
 [Link] returns 3, indicating it’s a 3D array.
 If we want an array with more than two dimensions, we can add more 1s in the shape. Each
additional 1 adds another dimension. For example, using shape (1, 3, 3) creates a 3-dimensional
array, while (1, 1, 3, 3) creates a 4-dimensional array.

7. Array Arithmetic Operations

Example:
arr + 5
Explanation:
 Adds 5 to every element in the array.
Example:
a = arr + 5
a*2
Explanation:
 First, arr is added to 5, then every element in a is multiplied by 2.

8. Identity Matrix
Example:
[Link](3)
Explanation:
 Creates a 3×3 matrix with 1s along the diagonal (top-left to bottom-right) and 0s elsewhere.
Example:
[Link](4)
Explanation:
 Same concept but for a 4×4 matrix.

9. Random Module in Python

Example:
import random
 Explanation:
 Imports the random module for generating random numbers.
Example:
[Link]([1, 2, 3, 4, 5]) #pick any random number such as 3
Explanation:
 Picks a random number from the list [1, 2, 3, 4, 5].
Example:
[Link](1, 10)
Explanation:
 Generates any one random integer between 1 and 9.
Example:
[Link]()
Explanation:
 Generates a random decimal number between 0 (included) and 1 (excluded).
Example:
lis=[1,2,3,4,5]
[Link](lis)
lis #[5,1,3,2,4]
 This numbers can be shuffled in any position

10. Random Numbers Using NumPy

Example:
[Link].random_sample((5))
Explanation:
 Creates a 1D array with 5 random decimal numbers between 0 and 1.
Example:
arr=[Link](1, 5, size=(3, 4))
Explanation:
 Generates a 3×4 matrix with random integers between 1 and 4.

11. Reshaping Arrays

Example:
arr=[Link](1, 5, size=(3, 4))
[Link](2, 6)
Explanation:
 Changes the shape of arr into 2 rows and 6 columns, but keeps the same data.
 Before reshaping, make sure that size should be same by multiplying the rows and columns
Example:
arr=[Link](1, 5, size=(3, 4))
[Link](-1, 4)
Explanation:
 The -1 tells NumPy to automatically adjust the number of rows based on the total elements. We can
use -1 for both row and column.
Example:
arr=[Link](1, 5, size=(3, 4))
[Link](1,3,4)
 Resizing the dimension, this time it is 3 dimension, but make sure that multiplication should be
equals to arr size

12. Filtering Array with condition

Example:
arr1 = [Link](1, 10, (5, 6))
arr1 > 3 # this returns array with True or False
 arr1[arr1 > 3] #output: array([5, 4, 5, 8, 6, 5, 8, 8, 8, 7, 4, 6, 4, 4, 4, 5, 5, 4, 4, 4, 7, 5,
6], dtype=int32)
Explanation:
 Filtering values greater than 3
 format of mentioning condition arr1[condition]

13. How to access array using array indexing or slicing concept

Example:
arr1 = [Link](1, 10, (5, 6))
arr1
output:
array([[1, 3, 2, 9, 7, 1],
[3, 4, 4, 3, 4, 9],
[1, 6, 3, 6, 5, 8],
[6, 2, 6, 5, 8, 4],
[4, 5, 7, 4, 8, 4]], dtype=int32)
Example:
arr1[0] # fetch First row
output: array([1, 3, 2, 9, 7, 1], dtype=int32)
Example:
arr1[0:3] # slice from Rows 0 to 2, row 3 is excluded
output:
array([[1, 3, 2, 9, 7, 1],
[3, 4, 4, 3, 4, 9],
[1, 6, 3, 6, 5, 8]], dtype=int32)
Example:
arr[0][3] #access specific element between row 0 and column 3, alternative method: arr1[1, 2] works
same
output: np.int32(1)
Example:
arr1[0:3, [0, 2]] #get specific column from selected rows. get rows 0,1,2 and only columns 0 and 2 from
those rows(0,1,2)
output:
array([[1, 2],
[3, 4],
[1, 3]], dtype=int32)
Example:
arr1[0:3, 1:4] #Slicing both rows and columns using ranges. get rows 0,1,2 and columns 1,2,3 from those
rows(0,1,2)
output:
array([[3, 2, 9],
[4, 4, 3],
[6, 3, 6]], dtype=int32)

 In NumPy, when working with arrays like arr1, you can slice or select data in two main ways:
a. Using a range (colon syntax):
arr1[start_row:end_row, start_col:end_col]
o The colon (:) creates a range — it includes the start index but excludes the end index.
o It's useful when you want a continuous block of rows or columns.
o Example: arr1[1:4, 2:5]
o Get rows 1, 2, 3 (stop before 4)

o Get columns 2, 3, 4 (stop before 5)

 b. Using square brackets with a list [] (index selection)
arr1[row_range, [col1, col2, colN]]
o This lets you pick specific columns that may not be in a continuous sequence.

o The square brackets [] contain a list of exact indexes you want to fetch.
o Example: arr1[0:3, [1, 4]]
o Get rows 0, 1, 2
o Get only columns 1 and 4 (non-consecutive)

14. Math on Arrays

Example:
arr1 = [Link](1, 3, (3, 3))
arr2 = [Link](1, 3, (3, 3))
arr1 + arr2 # Add elements index wise
arr1 - arr2 #subtract elements index wise
arr1 * arr2 # Multiply element index wise
arr1 / arr2 # Divide elementwise

15. Matrix Multiplication

Example:
arr1 @ arr2
Explanation:
 Performs matrix multiplication, different from index wise multiplication (*).
 Condition for matrix multiplication arr1 @ arr2

here no of columns of matrix A=no of rows of matrix B

b=b then only matrix multiplication will be apply
result matrix will be shape of a*c

Example:
[Link](arr1, arr2)
Explanation:
 Another way to do matrix multiplication.

16. Broadcasting in NumPy

 Example:
arr + 5
Explanation:
 NumPy automatically adds 5 to each element, even though the array is multi-dimensional.
Example:
arr + [Link]([1, 2, 3, 4])
Explanation:
 NumPy expands the smaller array to match the shape of arr, then adds elements index-wise.

17. Transposing Arrays

Example:
a=[Link]([[1,2,3,4,5]])
a #output: array([[1, 2, 3, 4, 5]])
a.T
output:
array([[1],
[2],
[3],
[4],
[5]])
Explanation:
 Converts rows to columns and columns to rows.
Numpy Advance 2
5. Create Random Integer Arrays
Example:
import numpy as np

arr1 = [Link](1, 3, (3, 3))

arr2 = [Link](1, 3, (3, 3))
Explanation:
 Create two 3×3 arrays with random integers between 1 (inclusive) and 3 (exclusive).

6. Flattening an Array
Example:
[Link]()
Explanation:
 Converts the multi-dimensional array(arr1) into a 1D array (flattens it).

7. Expanding Dimensions
Example:
arr = [Link]([1, 2, 3, 4])
np.expand_dims(arr, axis=0) #output: array([[1, 2, 3, 4]])
Example:
np.expand_dims(arr, axis=1)
output:
array([[1],
[2],
[3],
[4]])
Explanation:
 axis=0: Adds a new row dimension → shape becomes (1, 4)
 axis=1: Adds a new column dimension → shape becomes (4, 1)

4. Zeros with Shape and Dimension Expansion

Example:
arr = [Link]((3, 4))
output:
rray([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])
Example:
np.expand_dims(arr, axis=0)
output:
array([[[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]]])
Example:
np.expand_dims(arr, axis=1)
output:
array([[[0., 0., 0., 0.]],
[[0., 0., 0., 0.]],
[[0., 0., 0., 0.]]])
Explanation:
  [Link]((3, 4)) creates a 3×4 array filled with zeros
 expand_dims adds a new row or column dimension

5. Squeeze to Remove Extra Dimensions

Example:
a = [Link]([[1], [2], [3]])
output:
array([[1],
[2],
[3]])
Example:
[Link](a) #output: output: array([1, 2, 3])
Explanation:
 Removes dimensions with size 1
 Transforms shape (3, 1) to (3,)

6. Repeat Elements (no IMP)

Example:
a = [Link]([[1], [2], [3]])
[Link](a, 4) # output: array([1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3])
Explanation:
 Here every element repeating 4 times, 1 repeating four times, 2 repeating four times, 3 repeating four
times
Example:
arr1=[Link](1,4,(3,3))
arr1
output:
array([[3, 1, 2],
[2, 3, 1],
[1, 2, 2]], dtype=int32)
Example:
[Link](arr1, 2) #output: array([2, 2, 2, 2, 1, 1, 2, 2, 2, 2, 1, 1, 2, 2, 2, 2, 1, 1])
[Link](arr1, 2, axis=0)
output:
array([[3, 1, 2],
[3, 1, 2],
[2, 3, 1],
[2, 3, 1],
[1, 2, 2],
[1, 2, 2]], dtype=int32)
Explanation:
 Here, every row element is repeating twice, row 1 [3,1,2] repeating twice.
 In the same way other rows repeated twice
Example:
[Link](arr1, 2, axis=1)
output:
array([[3, 3, 1, 1, 2, 2],
[2, 2, 3, 3, 1, 1],
[1, 1, 2, 2, 2, 2]], dtype=int32)
Explanation:
 The original shape of arr1 is (3, 3) — 3 rows and 3 columns.
 [Link](..., axis=1) means we are repeating along the columns.
 For each element in a row, the value is repeated twice, one after the other.
  Final shape becomes (3, 6) because each of the 3 columns becomes 2 repeated columns.
 Row-wise breakdown:
Row 1: [3, 1, 2] → [3, 3, 1, 1, 2, 2]
Row 2: [2, 3, 1] → [2, 2, 3, 3, 1, 1]
Row 3: [1, 2, 2] → [1, 1, 2, 2, 2, 2]

7. Shifting Array Elements (no IMP)

Example:
a = [Link]([[1], [2], [3]])
[Link](a, -1)
c=[Link]([[1,2],[3,4],[5,6],[7,8]])
[Link](c, 2, axis=0)
Explanation:
 [Link] moves elements in the array by a specified number of steps
 Negative values roll elements left/up
 Positive values roll elements right/down

8. Arithmetic Operations
Example:
arr1 + arr2
arr1 - arr2
arr1 * arr2
arr1 > arr2
~arr1
Explanation:
 Performs element-wise operations: addition, subtraction, multiplication, comparison, and bitwise
NOT

9. String Operations
Example:
d = [Link](["pw", "skills"])
[Link](d)
[Link](d)
Explanation:
 String operations on NumPy arrays—convert to uppercase or capitalize the first letter

10. Math Functions

Example:
arr1=[Link](1,4,(3,3))
[Link](arr1)
np.log10(arr1)
[Link](arr1, 3)
[Link](arr1)
[Link](arr1)
Explanation:
 Apply math functions element-wise—sin, base-10 logarithm, power, and compute mean and
standard deviation

11. Sorting and Searching

Example:
e = [Link]([5, 6, 1, 2])
[Link](e) #output: array([1,2,5,6])
f = [Link]([5, 199, 200, 6, 7, 8])
[Link](f, 170)
np.count_nonzero(f) GiGives the
[Link](f > 0) # f [f > 0] this fetch index number where greater than 0 value is present same result
[Link](f > 0, f)
Explanation:
 [Link]() - Sort arrays
 [Link]() - Find insert position to keep elements in order. Find indices where elements
should be inserted to maintain order.
 np.count_nonzero() – Get count of non zero values
 [Link]() - Get values based on conditions

12. Byte Swap

Example:
[Link]()
Explanation:
 Reverses the byte order in the array—useful in specific data handling or system compatibility
scenarios

13. Matlib Arrays

Example:
import [Link] as nm

[Link](5)
[Link]((3, 4))
Explanation:
 Create arrays filled with zeros or ones using the matlib module

14. Linear Algebra (Optional)

Example:
arr1 @ arr2
[Link](arr1)
[Link](arr2)
[Link](a, b) #here matrix “a” have (x,y,z) values while b have (c) x+y+z=c
Explanation:
 @: Matrix multiplication
 det: Compute the determinant of a matrix
 inv: Calculate the inverse of a matrix
 solve: Solve a system of linear equations
Pandas Basic
1. Introduction to Pandas
 Pandas is a powerful tool for analyzing and modifying data in Python.
 It helps work with structured data like tables or spreadsheets.
 Created by Wes McKinney in 2008.

2. Reading a CSV File

Example:
import pandas as pd
df = pd.read_csv("[Link]")
df
Explanation:
 pd.read_csv() loads a CSV file and turns it into a DataFrame (table format).
 By default, Pandas assumes the first row contains column names.

3. Ignoring Column heading Names

Example:
pd.read_csv("[Link]", header=None)
Explanation:
 This tells Pandas not to treat the first row as column heading names.
 Instead, Pandas assigns integer index values to rows and columns.

4. Skipping Rows
Example:
pd.read_csv("[Link]", skiprows=2)
Explanation:
 Skips the first two rows while loading the data.

5. Selecting Specific Columns

Example:
pd.read_csv("[Link]", usecols=['program_id', 'application_process'])
Explanation:
 Loads only the selected columns instead of reading the whole file.

6. Data Structures in Pandas

Example:
 Series: 1-dimensional (like a single list of values).
 DataFrame: 2-dimensional (rows and columns, made up of multiple Series).

7. Checking Data Type of DataFrame

Example:
type(df)
Explanation:
 This checks the type of df and confirms that it is a DataFrame.

8. Accessing a Single Column

Example:
df["application_process"]
df.application_process
Explanation:
 Both methods return the "application_process" column as a Series.
9. Checking Column Type
Example:
type(df["application_process"])
Explanation:
 Since columns in a DataFrame are Series, this confirms its data type.
 Each columns are seriess

10. Creating a Pandas Series

Example:
l = [1, 2, 3, 4]
s = [Link](l)
s
Explanation:
 A Series is created from a list [1, 2, 3, 4].
 Default index values are integers starting from 0.

11. Accessing Data in a Series

Example:
l = [1, 2, 3, 4]
s = [Link](l)
s[0] # First element
s[1:] # Elements from index 1 onwards
s[2:4] # Elements from index 2 to 3
Explanation:
 Retrieves specific elements from the Series using indexing.

12. Custom Index for a Series

Example:
d = [Link]([100, 200, 300], index=["Ajay", "Bijay", 1])
d
output:
Ajay 100
Bijay 200
1 300
dtype: int64
Explanation:
 We can define custom labels for rows instead of using default numbers.
 We can also do custom indexing for Data Frame.

13. Accessing Data in Custom Indexed Series

Example:
d["Ajay"]
[Link]
Explanation:
 d["Ajay"] retrieves the value 100.
 [Link] shows all defined index labels.

14. Reset Index in Series

Example:
d.reset_index(drop=True)
Explanation:
 Removes custom indexing and restores default numeric indexing.
 This doesn’t change in original series instead it creates a new series. To change in the original series
just write “inplace=True”
 This works for both data frames and series

15. Convert Series to DataFrame

Example:
[Link](d) #d is series from previous example
Explanation:
 Converts a Series into a table-like DataFrame.

16. View First Few Rows of DataFrame

Example:
[Link]() #view first 5 rows by default
[Link](10) # First 10 rows
[Link](2) # First 2 rows
Explanation:
 head() helps preview the first few rows of data.

17. View Last Few Rows of DataFrame

Example:
[Link]() #view last 5 rows by default
[Link](2)
Explanation:
 tail() helps preview the last few rows of data.

18. Check Shape of DataFrame

Example:
[Link]
Explanation:
 Returns (number of rows, number of columns).

19. See Column Names

Example:
[Link]
list([Link]) #convert all column into list
Explanation:
 Shows all the column names in the DataFrame.

20. Sampling & Getting Data Summary

Example:
[Link](2) # get any Random 2 rows
[Link]() # Summary of data
[Link] # Data types of each column
Explanation:
 sample() picks random rows.
 info() provides an overview of the DataFrame.
 dtypes shows the type of each column.

21. Add a New Column

Example:
d["new_column"] = "Ajay"
d[“new_column1”]=[1,2,3]
Explanation:
 Creates a new column named "new_column" in the DataFrame and add “Ajay” as value in all
rows.(Add constant values as ‘Ajay’ in all rows.)
 To prevent constant values just add the number of values in square brackets and number of values is
dependent on number of rows.

22. Rename Columns

Example:
[Link] = ["col1", "col2", "col3"]
Explanation:
 Updates column names from old names to new ones.

23. Access Multiple Columns

Example:
df_subset = df[["name", "status", "languages"]]
Explanation:
 Square brackets [] return multiple columns.

24. Read Excel & Other File Formats

Example:
df1 = pd.read_excel("LUSID Excel - Setting up your market [Link]")
df3 = pd.read_csv("[Link]
Explanation:
 Reads data from an Excel file.
 Loads a CSV file from an online link.

25. Extract Tables from Web Pages

Example:
url_df = pd.read_html("[Link]
df4 = url_df[0]
Explanation:
 Extracts tables from a webpage.

26. Save Data to CSV

Example:
df4.to_csv("[Link]", index=False)
Explanation:
 Saves the DataFrame as a CSV file.
 index=False prevents saving row numbers.
 By default, pandas includes the index as the first column in the output
file. Setting index=False removes this column, resulting in a cleaner CSV file that contains only the
data and header row.

27. Read JSON Data

Example:
url = "[Link]
pd.read_json(url)
Explanation:
 Loads JSON data from a web API directly into a DataFrame.

28. Fetch JSON Using Requests

Example:
import requests
data = [Link](url)
[Link]()
Explanation:
 Fetches JSON data using the requests library.
Important functions under pandas:

📌 General & Creation

 [Link]()
 [Link]()

📌 Data Inspection
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]
 [Link]
 [Link]
 [Link]
 [Link]

📌 Data Selection & Filtering

 df['col']
 df[['col1', 'col2']]
 [Link][]
 [Link][]
 [Link][]
 [Link][]
 [Link]()

📌 Sorting & Ranking

 df.sort_values()
 df.sort_index()
 [Link]()

📌 Missing Data Handling

 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()

📌 Aggregation & Grouping

 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 📌 Merging & Joining
 [Link]()
 [Link]()
 [Link]()

📌 Reshaping
 [Link]()
 df.pivot_table()
 [Link]()
 [Link]()
 [Link]()
 [Link]()

📌 String Functions
 [Link]()
 [Link]()
 [Link]()
 [Link]()
 [Link]()

📌 Datetime Functions
 pd.to_datetime()
 df['col'].[Link]
 df['col'].[Link]
 df['col'].[Link]
 df['col'].[Link]

📌 Data Export/Import
 pd.read_csv()
 df.to_csv()
 pd.read_excel()
 df.to_excel()
 pd.read_json()
 df.to_json()

Pandas Advance 1
1. Importing the Required Library
Example:
import pandas as pd
Explanation:
 This imports the pandas library, which helps us manage and analyze datasets in Python.

2. Loading the Titanic Dataset

Example:
df = pd.read_csv("[Link]
Explanation:
 This command reads the Titanic dataset from a public URL and stores it in a variable called df
(short for "dataframe").

3. Displaying the Dataset

Example:
df
Explanation:
 Shows the entire dataset in tabular format.

4. Checking Column Names

Example:
[Link]
Explanation:
 Lists all column names available in the dataset.

5. Viewing the First Few Rows

Example:
[Link]()
Explanation:
 Displays the first 5 rows to give a quick look at the dataset.

6. Viewing the Last Few Rows

Example:
[Link]()
Explanation:
 Shows the last 5 rows of the dataset.

7. Checking Column Data Types

Example:
[Link]
Explanation:
 Displays the data type of each column (e.g., numbers, text, etc.).

8. Getting Detailed Information About Dataset

Example:
[Link]()
Explanation:
 Provides a summary of the dataset, including column names, non-null values, and data types.

9. Statistical Summary of Numeric Columns

Example:
[Link]()
Explanation:
 Displays statistical data such as minimum, maximum, average, and quartiles for numeric columns.
  There are two types of column data: numerical(integer and float) and categorical(string/object/text)
data.
 Numerical data consists of values: continuous(float) and discrete(integer)

10. Five-Point Summary of Numeric Data

Explanation:
A summary including:
 Minimum value
 25% quartile
 50% median
 75% quartile
 Maximum value

11. Selecting Specific Columns

Example:
df[["PassengerId", "Survived", "Pclass"]]
Explanation:
 Extracts and displays only the selected columns.

12. Identifying Columns Containing Text Data

Example:
[Link] == "object" # here object is a string data type
Explanation:
 Checks which columns are stored as text Sring data.

13. Extracting Only Text-Based Columns

Example:
[Link][[Link] == "object"] #fetching only column names with object(text) data types
output:
Name object
Sex object
Ticket object
Cabin object
Embarked object
dtype: object

Example:
[Link][[Link] == "object"].index #fetching only those column header which are object data types
output:
Index(['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked'], dtype='object')
Example:
df[[Link][[Link] == "object"].index] # Displays only the columns containing text data, This works
similarly to list indexing, e.g., lst[index_no] like lst=[“ritesh”,True,12.4,13,None]
lst[1] #output: True

14. Extracting Numeric Columns

Example:
df[[Link][[Link] != "object"].index]
Explanation:
 Displays only the columns containing numerical data.

15. Statistical Description of Numeric Data

Example:
df[[Link][[Link] != "object"].index].describe()
Explanation:
 Provides statistical details only for numeric columns.

16. Statistical Description of Text Data

Example:
[Link](include="object")
Explanation:
 Provides insights into categorical (text-based) columns only, it will not show insights to numerical
column.
 If we want both numerical and categorical data just include=”all”
Example: If we want to get statistical information(count, unique, top, freq) for numerical columns also, so
we will first convert numeric columns into object(categorical) data type.
[Link](object).describe()
Explanation:
 By converting all columns to object type, we can use describe() to get text statistics (like unique
values and most common value) for all columns, including those that were originally numeric.

17. Shape of the Dataset

Example:
[Link]
Explanation:
 Displays the number of rows and columns in the dataset.

18. Slicing Specific Rows

Example:
df[0:100]
Explanation:
 Shows rows 0 to 99 from the dataset.

19. Skipping Rows While Selecting

Example:
df[0:100:5]
Explanation:
 Displays every 5th row from the first 100 rows.

20. Adding a New Column with Fixed Values

Example:
df["new_col"] = "pwskills"
Explanation:
  Adds a new column called new_col with the same value ("pwskills") for every row.

21. Creating a 'Family' Column

Example:
df["family"] = df['SibSp'] + df['Parch']
Explanation:
 Creates a new column family by adding SibSp (siblings/spouses) and Parch (parents/children) to
show total family members aboard.

22. Different Categories of sub-class

Example:
[Link](df['Pclass'])
output:
[3, 1, 3, 1, 3, ..., 2, 1, 3, 1, 3]
Length: 891
Categories (3, int64): [1, 2, 3]
Explanation:
Segregates (extracts) unique values in a column and treats them as categories

23. Unique Values in Specific Columns

Example:
df[‘Plass’].unique()
output:
array([3, 1, 2])
Example:
df['Sex'].unique()
output:
array(['male', 'female'], dtype=object)
Example: count unique values
df['Sex'].nunique() #nunique() function counts the unique values within a single column
output:
#male and female are two unique valuess
Explanation:
 Displays unique values found in Each column.

24. Counting Occurrences of Each Value

Example:
df['Sex'].value_counts()
output:
male 577
female 314
Name: count, dtype: int64
Explanation:
 Shows how many times each gender appears in the dataset.

25. Percentage Distribution of a Column

Example:
df['Sex'].value_counts(normalize=True) * 100
output:
male 64.758698
female 35.241302
Name: proportion, dtype: float64
Explanation:
 Displays the percentage of number of males and females.
26. Finding Passengers Below Age 5
Example:
df[df['Age'] < 5]
Example: How many passengers are less than 5 years old
df[df[“Age”]<5].shape #output: (40,14) here there are 40 rows this means there are 40 passenger less than
5 years
boBoth are two different methods, but works same.
len(df[df['Age'] < 5]) #output: 40
Find number of passengers
Example: Names of children less than 5 years old
df[df['Age'] < 5]['Name']
df[df['Age'] < 5].Name GiGives the same result
Explanation:
 Filters passengers who are younger than 5 years old.

27. Counting Passengers

Example: How many passengers have paid less than average fare
len(df[df['Fare'] < df['Fare'].mean()]) #outupt: 680

Example: How many passengers have paid 0 fare

len(df[df["Fare"]==0]) #output: 15

Example: how many passengers are of class 1

len(df[df['Pclass'] == 1]) #output: 216

Example: How many passengers are male

len(df[df['Sex'] == "male"]) # 577

Example: How many females paid more than avg fare?

average_fare=df["Fare"].mean()
passenger_paid_more_than_fare=df["Fare"]>average_fare
len(df[(df["Sex"]=="female")&(passenger_paid_more_than_fare)]) #output: 104

Explanation:
 Counts the number of passengers

28. Finding Maximum and Minimum Fare

Example:
max(df['Fare']) #output: 512.3292
min(df['Fare']) #output: 0.0
Explanation:
 Finds the highest and lowest fare paid.

29. Finding Passengers Who Paid Maximum Fare

Example:
df[df['Fare'] == max(df['Fare'])].Name
Explanation:
 Lists names of passengers who paid the highest fare.

30. Average Age of People Who Paid Maximum Fare

Example:
# Step 1: Find the maximum fare
max_fare = df['Fare'].max()
# Step 2: Filter rows where fare is equal to max_fare
max_fare_rows = df[df['Fare'] == max_fare]

# Step 3: Calculate average age (excluding NaN)

average_age = max_fare_rows['Age'].mean()

print("Maximum Fare:", max_fare)

print("Average Age of people who paid max fare:", average_age)

Explanation:
 Finds the average age of passengers who paid the highest fare.

Pandas Advance 2

1. Importing Pandas Library

Example:
import pandas as pd
Explanation:
 pandas is a powerful library used for data manipulation and analysis.
 import pandas as pd allows us to use Pandas functionalities with the alias pd.

2. Loading the Titanic Dataset

Example:
df = pd.read_csv("[Link]
Explanation:
 Reads a CSV file from a URL and loads it into a Pandas DataFrame (df).
 read_csv() is a function used to import structured data.

3. Viewing Data
Example:
[Link]()
[Link]()
Explanation:
 head() displays the first 5 rows of the dataset.
 tail() displays the last 5 rows of the dataset.

4. Basic Information About Data

Example:
[Link]
[Link]()
[Link]
Explanation:
 shape returns the number of rows and columns.
 info() displays metadata, including column names, data types, and missing values.
 dtypes gives the data types of each column.

5. Selecting Specific Rows

Example:
df[0:100]
df[0:100:2]
df[0:100:5]
Explanation:
 df[0:100] selects the first 100 rows.
 df[0:100:2] selects every second row from the first 100 rows.
 df[0:100:5] selects every fifth row from the first 100 rows.

6. Difference Between iloc and loc

iloc (Integer Location Indexing)
 Uses numeric positions to locate rows and columns.
 implicit/integer/internal indexing
 Example: [Link][0:3] selects the first three rows (index 0 to 2).
loc (Label-Based Indexing)
 Uses named indexes (row labels or column names).
 explicit or named indexing
 Example: [Link][0:3] selects rows labeled 0, 1, 2, and 3 (not position-based).
Key difference? iloc works with numbers, loc works with labels/names.

7. Selecting Data Using Indexing

Example:
[Link][0:3]
[Link][0:3]
Explanation:
 iloc (Integer Location) selects rows based on index positions.
 loc (Location) selects rows based on labels (named index).
 [Link][0:3] selects rows from index 0 to 2.
 [Link][0:3] selects rows with labels 0, 1, 2, and 3.

8. Incorrect Usage of iloc

Example:
[Link][0:2, ["Name", "Sex"]] # This throws an error
[Link][0:2, ["Name", "Sex"]] # This won’t throw an error and result will be printed
Explanation:
 iloc only accepts integer positions, but ["Name", "Sex"] are column names.
 Instead, loc should be used to reference column names.

9. Selecting Columns by Index

Example:
[Link][0:2,3:6]
Explanation:
 Selects rows from index 0 to 1 and columns from index 3 to 5.
 Uses integer indexing.

10. Creating a List from a Column

Example:
list(df['Name'][2:5])
Explanation:
 Extracts rows 2 to 4 from the "Name" column and converts them into a Python list.

11. Creating a Pandas Series

Example:
s = [Link](list(df['Name'][2:5]), index=['a', 'b', 'c'])
s
Explanation:
 Creates a Series from the "Name" column, assigning custom index labels 'a', 'b', 'c'.

12. Concatenation Behavior

Example:
s1 = [Link](list(df['Name'][5:8]))
s+s1
Explanation:
 s+s1 fails because their index values are different.
 Index values must match for successful concatenation.
Example:
s1 = [Link](list(df['Name'][5:8]) , index=['a', 'b', 'c'])
s+s1

Explanation:
 Now, this time both two different series has been concatenated because both series has same indexes

13. Dropping Columns

Example:
[Link]('PassengerId', axis=1, inplace=True)
 Explanation:
 Removes the "PassengerId" column (axis=1 indicates column removal).
 If PassengerId would be in rows then axis=0 need to mention (axis=0 indicates row removal).
 inplace=True modifies the original data frame.
 If we do not use inplace=True then it won’t effect original data frame (df) instead in absence of
inplace=True a new data frame will be created.
Example:
[Link](1, inplace=True)
Expalantion:
 This drop row 1 and inplace=True will affect the original data frame.

14. Resetting Index

Example:
df.reset_index(drop=True)
df.reset_index(inplace=True)
Explanation:
 Resets row indexes and removes the previous index(this were the previous index: , index=['a', 'b',
'c']), creates a new index but changes won’t effect the original data frame.
 Here inplace=True modifies the original data frame

15. Setting a New Index

Example:
df.set_index('Name', inplace=True)
Explanation:
 Sets "Name" as the new index.

16. Extracting Specific Rows Using loc

Example:
[Link]['Braund, Mr. Owen Harris']
Explanation:
 Retrieves details for a passenger with index "Braund, Mr. Owen Harris".
Example:
df.reset_index() #if we don’t pass drop=True then the Name(set as new index) will by default convert into
column

17. Checking Missing Values

Example:
df1 = pd.read_csv('[Link]') #reading csv file
[Link]().sum()
output:
taxonomy_id 0
name 0
parent_id 11
parent_name 11
dtype: int64

Explanation:
 Counts missing values in each column.

Why checking null values?

 In machine learning we check null values because we need to drop the null values.
 Machine learning is all about learning patterns from the data.
  If there is null value then it will not help in learning the patterns from the data instead it will throw
an error, so you want to delete this null values.
 Instead of deleting null values we can also do imputation.
 Imputation means replacing null values with some values and those some values are:
a. if data is numerical, replace null values with mean and meadian.
b. if data is categorical, replace null values with mode
c. else replace with constant
NOTE: if outlier has been remove then we replace null value with mean but if outlier is not remove
then we replace null values with median.
what is outlier?
a. if there are values such as 1,2,3,4,5,100,etc..
b. in this sequence 100 is extreme values(high or low values occur at the last or start) and this
is known as outlier.
c. outlier is those value that is suddenly disturb the sequence

18. Dropping Missing Values

Example:
[Link]()
[Link](axis=1)
df1[["name"]].dropna(axis=1)
Explanation:
 dropna() removes rows containing missing/null values.
 dropna(axis=1) removes columns containing missing values.
 df1[["name"]].dropna(axis=1) this drop the null value for one column and here double square bracket
means it is a dataframe

19. Imputing(replacing) Missing Values

Example:
[Link](0)
[Link]("somevalue")
Explanation:
 fillna(0) replaces all missing values with 0.
 "somevalue" fills missing values with a placeholder.

20. Replacing missing values with mean and median

Example:
data = {
'A': [1, 2, None, 4, 5, None, 7, 8, 9, 10],
'B': [None, 11, 12, 13, None, 15, 16, None, 18, 19]
}
df2=[Link](data)
df2
output:

Example:
mean_value=df2["A"].mean() #mean of column “A”
df2["A"].fillna(mean_value) #replacing all null/missing values in column “A” with mean
output:
0 1.00
1 2.00
2 5.75
3 4.00
4 5.00
5 5.75
6 7.00
7 8.00
8 9.00
9 10.00
Name: A, dtype: float64

Example:
median_value=df2["A"].median() #median of column “B”
df2["B"].fillna(median_value) #replacing all null/missing values in column “B” with median
output:
0 6.0
1 11.0
2 12.0
3 13.0
4 6.0
5 15.0
6 16.0
7 6.0
8 18.0
9 19.0
Name: B, dtype: float64

21. Forward and Backward Fill

Example:
[Link](method="ffill")
output:

[Link](method="bfill")
output:

Explanation:
 Forward fill (ffill) start seeing from bottom of data frame and fills missing values from the previous
row value.
Eg. there are 4 rows and in third row there’s missing value so forward fill will copy the second row
value and fill in third row.
 Backward fill (bfill) start seeing from top of data frame and fills missing values using the next row
value.
 Eg. there are 4 rows and in 1st row there’s missing value so backward fill will copy the second row
value and fill in first row.

22. Checking Duplicates

Example:
[Link]().sum() #output: 0
Explanation:
 Counts duplicate rows in df2.
 Here there’s no duplicate values therefore sum of duplicate values is zero

23. Analyzing Survival Rates

Example: What is the average fare paid by the people who survive or didn’t survive?
df = pd.read_csv("[Link]
df[df['Survived'] == 1]['Fare'].mean()
df[df['Survived'] == 0]['Fare'].mean()
Explanation:
 Finds the average fare paid by survivors and non-survivors.

24. Grouping Data

 groupby is a powerful feature in pandas used to split your data into groups based on a column, apply
some operation (like mean, sum, count, etc.), and then combine the results.
 This is known as the Split–Apply–Combine strategy.
 groupby lets you perform calculations per group within your data

When to Use groupby:

 When your data has categories (like cities, genders, product types).
 When you want to calculate things within each group, like:
o average salary per department
o total sales per product
o standard deviation by region

Example:

[Link]('Survived').mean(numeric_only=True)
output:

Explanation:
 Groups by "Survived" and computes average values.
 numeric_only=True this calculates for numeric column only
Example:
[Link](["Survived"])["Fare"].agg([min,max,[Link],'var',sum,'mean','median'])
output:

Explanation:
 [Link](["Survived"]): Splits the DataFrame into groups based on the values in the "Survived"
column — typically 0 (did not survive) and 1 (survived).
 ["Fare"]: Selects only the "Fare" column for analysis within each group.
 .agg([...]): Applies multiple aggregation functions (like min, max, mean, etc.) to the "Fare" column
for each group.
Result: A summary table showing various statistics of "Fare" for survivors and non-survivors.

1. Aggregating Survival Data

Example:
[Link](["Gender","Pclass"])["Survived"].sum().to_frame()
Explanation:
 [Link](["Gender", "Pclass"])
Groups the DataFrame by combinations of Gender and Pclass.
 ["Survived"]
Selects the Survived column for aggregation.
 .sum()
Sums the Survived values within each group.
Since Survived is typically 0 (died) or 1 (lived), this gives the number of survivors in each group.
 .to_frame()
Converts the result (a Series) back into a DataFrame.
Example:
[Link](['Sex', 'Pclass'])['Survived'].sum().unstack()
Explanation:
 Here unstack() It converts row index levels into columns.

 Moves Pclass from row index to column headers.

 Makes the output easier to read as a table
 Pandas Advance 2
1. Importing Required Libraries
Example:
import pandas as pd
import numpy as np
Explanation:
 pandas is imported as pd: Used for handling data in tabular form (DataFrames).
 numpy is imported as np: Useful for numerical computations.

2. Reading Data from a URL

Example:
df = pd.read_csv("[Link]
Explanation:
 This line reads a CSV file from the internet into a DataFrame df.
 The dataset used is the famous Titanic dataset.

3. String Concatenation
Example:
"pw" + "skills"
Explanation:
 Concatenates two strings and returns 'pwskills'.
 Similarly concatenation between two data set is not possible using “+” operator for that we need to
use function such as concat.

4. Selecting Specific Columns and Rows

Example:
df1 = df[["Name", "Sex", "Age"]][0:5]
output:

Explanation:
 Selects columns "Name", "Sex", and "Age" from df.
 Then takes the first 5 rows (index 0 to 4) and stores in df1.

5. Select Next 5 Rows into Another DataFrame

Example:
df2 = df[["Name", "Sex", "Age"]][5:10]
output:
Explanation:
 Similar to df1, but this selects rows 5 to 9 and saves them into df2.

6. Vertical Concatenation
Example:
[Link]([df1, df2], axis=0)
output:

Explanation:
 Concatenates df1 and df2 row-wise (vertically).
 axis=0 means stacking rows on top of each other.
 By-Default axis=0 so we don’t need to explicitly mention axis=0

7. Horizontal Concatenation
Example:
[Link]([df1, df2], axis=1)
output:

Explanation:
 Concatenates df1 and df2 column-wise (side by side).
 axis=1 joins DataFrames by their row indices and extends the table by adding more columns

 If df1 and df2 don’t have the same number of rows or matching index labels, pandas fills in the
missing spots with NaN(Not a number) to keep the DataFrame rectangular (because every row
must have the same number of columns).

8. Resetting Index in df2

Example:
df2.reset_index(drop=True, inplace=True)
output:

Explanation:
 Resets the index of df2 to start from 0.
 drop=True means old index is removed.
 inplace=True updates df2 directly.

9. Horizontal Concatenation After Reset

Example:
[Link]([df1, df2], axis=1)
output:

Explanation:
 Same as step 7, but after ensuring both DataFrames have same indexes.
 This time both df1 and df2 have same indexing so no NaN values are printed.

10. Creating Two New DataFrames for Merge

Example:
df1 = [Link]({'key1':[1,2,4,5,6], 'key2':[4,5,6,7,8], 'key3':[3,4,5,6,6]})
output:

df2 = [Link]({'key1':[1,2,45,6,67], 'key4':[56,5,6,7,8], 'key5':[3,56,5,6,6]})

output:

Explanation:
 Two new DataFrames with common column key1(common elements row:1,2,6) are created.
 These will be used for merging based on keys.
11. Different Types of Merges(merge just combine based on common columns)

Suppose there are two data sets named A and B.

The following are the different types of merges/join operations that can be performed using these data sets:

a. Inner Merge (Intersection):

o Combines rows with matching values in both DataFrame A and B.
o Only rows with common keys are retained.
b. Left Merge:
o All rows from DataFrame A are preserved.
o Matching rows from DataFrame B are added where available.
o Non-matching B entries result in NaN.
c. Right Merge:
o All rows from DataFrame B are preserved.
o Matching rows from DataFrame A are added where available.
o Non-matching A entries result in NaN.
d. Outer Merge:
o Includes all rows from both DataFrames.
o Combines matching rows; fills missing values with NaN when no match is found.

Example:
[Link](df1, df2, how='inner')
output:

[Link](df1, df2, how='left')

output:

[Link](df1, df2, how='right')

output:
 [Link](df1, df2, how='outer')
output:

Explanation:
 inner: Return only those rows that have matching values in key1 from both DataFrames and exclude
the rows that do not match in both data frame.
 left: Return all rows from df1, and only matching rows from df2. If no match in df2 then return NaN
 right: Return all rows from df2, and only matching from df1. If no match in df1 then return NaN
 outer: Return all rows from both(df1 and df2), If no match then missing values filled with NaN.

12. Cross Merge(Cartesian Product)

o Produces every possible combination of rows from DataFrame A and B.
o If one DataFrame has 3 rows and the other has 4, the result will have 3 × 4 = 12 rows.

Example:
[Link](df1, df2, how="cross")
Explanation:
 Performs a Cartesian product (every row in df1 combined with every row in df2).
 The result have rows from 0 to 24.

13. Custom Merge Keys

Example:
[Link](df1, df2, how='left', left_on='key2', right_on='key4')
output:

Explanation:
 Merges based on custom columns: key2 from df1 and key4 from df2.

15. Creating DataFrames with Indexes for Join(join just combine based on common index)
Join is similar to merge so whatever the types and meaning of merge has, join also possessed the same type
and meaning. The only difference is it combine elements based on common index.
Example:
df1 = [Link]({'key1':[1,2,4,5,6], 'key2':[4,5,6,7,8], 'key3':[3,4,5,6,6]}, index=['a', 'b', 'c', 'd', 'e'])
output:
df2 = [Link]({'key6':[1,2,45,6,67], 'key4':[56,5,6,7,8], 'key5':[3,56,5,6,6]}, index=['a', 'b', 'h', 'i', 'j'])
output:

Explanation:
 Creates df1 and df2 with custom row labels (indexes).

15. Join Based on Index

Example:
[Link](df2, how='inner')
output:

[Link](df2, how='left')
output:

d
[Link](df2, how='right')
output:

[Link](df2, how='outer')
output:

[Link](df2, how='cross')

Explanation:
  join() matches rows using index.
 inner: Only rows with matching indexes in both.
 left: All from df1, matches from df2.
 right: All from df2, matches from df1.
 outer: All indexes from both DataFrames.
 cross: All combinations of index pairs.

16. Using apply() to Create New Column

Example:
df["Fare_inr"] = df['Fare'].apply(lambda x: x*90)
Explanation:
 Multiplies every value in "Fare" column by 90.
 Stores it in a new column "Fare_inr" (converting to INR).

17. Length of String

Example:
len("Ram")
Explanation:
 Returns 3, the number of characters in "Ram".

18. Creating Column with Length of Each Name

Example:
df["Name_len"] = df["Name"].apply(len)
Explanation:
 Computes the length of each name and stores it in a new column.

19. User-defined Function in apply()

Example:
def convert(x):
return x*90

df["Fare1"] = df["Fare"].apply(convert)
Explanation:
 Defines a function convert to multiply input by 90.
 Applies it to "Fare" column and stores in "Fare1".

20. Categorizing Fare into Flags

Example:
def create_flag(x):
if x < 10:
return "cheap"
if x >= 10 and x < 20:
return "medium"
else:
return "high"

df["flag_fare"] = df['Fare'].apply(create_flag)
Explanation:
 Defines create_flag function to label fares:
o "cheap" for < 10
o "medium" for 10–19.99
o "high" for ≥ 20
 Applies it to "Fare" column and stores in "flag_fare".
21. Set and Reset Index
Example:
data = {"a": [1, 2, 3, 4], "b": [5, 5, 6, 7], "c": ["pw", "skills", "aj", "cj"]}
df1 = [Link](data)
df1.set_index('c', inplace=True)
df1.reset_index(drop=True, inplace=True)
Explanation:
 Creates a DataFrame.
 Sets "c" column as index.
 Then resets the index, dropping the original index.

22. Reindexing Rows

Example:
[Link]([1, 2, 3, 0])
Explanation:
 Changes the order of rows based on the list [1, 2, 3, 0].

23. Iterating Over Rows and Columns

Example:
for i in [Link]():
print(i, "......")

for i in [Link]():
print(i)
Explanation:
 iterrows() iterate through through each row.
 items() iterate through each column.

24. Applying a Function Row-wise and Column-wise

Example:
def func_sum(x):
return [Link]()

[Link](func_sum, axis=0)
[Link](func_sum, axis=1)
Explanation:
 axis=0: Sums values column-wise.
 axis=1: Sums values row-wise.

25. Applying a Function to Every Element

Example:
[Link](lambda x: x**2)
Explanation:
 Squares every element in the DataFrame using applymap().

26. Another DataFrame Example

Example:
data = {"a": [100, 200, 13, 4], "b": [5, 5, 6, 7], "c": ["pw", "skills", "aj", "cj"]}
df2 = [Link](data)
Explanation:
 Creates a new DataFrame df2 with numeric and text data.

26. Another DataFrame Example

Example:
data = {"a": [100, 200, 13, 4], "b": [5, 5, 6, 7], "c": ["pw", "skills", "aj", "cj"]}
df2 = [Link](data)
Explanation:
 Creates a new DataFrame df2 with numeric and text data.

27. Sorting values

Example:
df2.sort_values(by=True)
Explanation:
sort_values() this sort the values inside data frame, by default ascending=True if we set
ascending=False then it will sort in descending order.
Pandas Advance 4
1. Creating a DataFrame with a description column
Example:
import pandas as pd

data={"description":["PW Skills is your one-stop-shop for upscaling. Get maximum value for timeand
resources you invest, with job-ready courses & high-technology,available at the lowest cost."]}
df1 = [Link](data)

df1
Explanation:
 Create a dictionary and convert it into data frame.

2. Setting column width to display long text properly

Example:
pd.set_option('display.max_colwidth', 1000)
df1
output:

Explanation:
 pd.set_option(...): Adjusts the display settings of pandas.
 'display.max_colwidth': Sets the max number of characters shown in a column.
 1000: Increases it so that the entire text is visible without truncation.

3. Setting max rows and columns to show

Example:
pd.set_option('display.max_rows', 100)
pd.set_option('display.max_columns', 100)
Explanation:
 Sets the number of rows and columns, pandas can display in one go.
 Helpful when working with large DataFrames.

4. Finding the length of a string

Example:
len("pw skills")
Explanation:
 len(...): Returns the number of characters in the string, including spaces.
 "pw skills" has 9 characters.

5. Adding a column with character length of each description

Example:
df1["char_len"] = df1['description'].apply(len)
df1
output:
Explanation:
 apply(len): Applies the len() function to each row of the 'description' column.
 Creates a new column 'char_len' to store the result.

6. Basic string operations

Example:
a = "I am Ajay"
len(a)

len([Link]())
Explanation:
 a = "I am Ajay": A string is assigned to variable a.
 len(a): Returns total characters in the string (including spaces).
 [Link](): Splits the string into words using space. Split the string and store it in list.
 len([Link]()): Counts the number of words.
 In the same way we can also count words in data frames.

7. Count number of words in the description(count words in data frame)

Example:
df1["word_count"] = df1["description"].apply(lambda x: len([Link]()))

df1
Explanation:
 Adds a new column word_count.
 Uses lambda x: len([Link]()) to count how many words are in each description.

8. Creating a new DataFrame and changing case

Example:
data = {'text':['Hello data science', 'I love ML', 'I read ml books']}
a = [Link](data)

a['text_lower'] = a['text'].[Link]()
a

a['text_upper'] = a['text'].[Link]()
a
Explanation:
 Creates a new DataFrame a with a text column.
 .[Link](): Converts all text to lowercase.
 .[Link](): Converts all text to uppercase.

9. Checking if a string starts with a specific letter

Example:
a['text'][0].startswith('H')
a['text'][1].startswith('H')
a['text'][1].startswith('I')
Explanation:
 .startswith('H'): Checks if the string begins with 'H'.
 Returns True or False accordingly.
10. Creating DataFrame and multiplying a column
Example:
df1 = [Link]({'a':[1,2,4,5,6], 'b':[4,5,6,7,8]})
df1

df1['mul_a'] = df1['a'] * 5
df1
Explanation:
 Creates DataFrame df1 with two columns: a and b.
 Creates new column mul_a by multiplying each value in a by 5.

11. Basic statistical functions

Example:
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
[Link]()
Explanation:
 .mean(): Average value
 .median(): Middle value
 .mode(): Most frequent value
 .min() / .max(): Minimum and maximum
 .sum(): Total of all values
 .var(): Variance
 .describe(): Summary of all basic statistics

12. Rolling mean (moving average)

Example:
df2 = [Link]({"a": [1, 2, 3, 4, 5, 6, 7, 8, 9]})
df2

[Link](window = 1).mean()
[Link](window = 2).mean()

df3 = [Link](window = 3).mean()

df3

[Link](inplace=True)
df3
Explanation:
 rolling(window=n): Computes a moving average using n values at a time.
 dropna(): Removes rows with missing (NaN) values after rolling operation.

13. Typecasting: convert float to int

Example:
df3['a_int'] = df3['a'].astype(int)

df3
Explanation:
 .astype(int): Converts data type of column a to integer.

14. Rolling window: sum, min, max

Example:
[Link](window = 2).sum()
[Link](window = 2).min()
[Link](window = 2).max()
Explanation:
 Applies sum, min, max to each rolling window of size 2.

15. Random group assignment

Example:
import numpy as np
[Link](['A', 'B'])

df2['Group'] = [Link](['A', 'B'], size = 9)

df2
Explanation:
 Randomly assigns 'A' or 'B' to each row in new column Group.

16. Grouping and rolling sum

Example:
[Link]('Group')['a'].rolling(window = 2).sum()
Explanation:
 Groups data by 'Group' column.
 Calculates rolling sum on column a within each group.

17. Cumulative sum

Example:
[Link]()
Explanation:
 Adds up values step-by-step, from top to bottom.

18. Working with date columns

Example:
df = [Link]({"date": ['2024-03-08', '2024-03-09', '2024-03-10']})
df

[Link]()
[Link]

df['updated_date'] = pd.to_datetime(df["date"])
[Link]

df['year'] = df['updated_date'].[Link]
df['month'] = df['updated_date'].[Link]
df['day'] = df['updated_date'].[Link]
Explanation:
 Converts string dates to datetime format.
 Extracts year, month, day from each date.

19. Creating date ranges

Example:
pd.date_range(start = '2024-07-10', end = '2025-07-10', freq = 'D')
pd.date_range(start = '2024-07-10', end = '2025-07-10', freq = 'H')
Explanation:
 Generates list of dates from start to end.
 freq='D': Daily; freq='H': Hourly.

20. Current timestamp and code execution time

Example:
[Link]()

start = [Link]()
for i in range(1000):
pass
end = [Link]()
print("Time taken for the execution of code", end-start)
Explanation:
 [Link](): Gets current date and time.
 Measures how long a block of code takes to run.

21. Measuring time using time module

Example:
import time
start = [Link]()
for i in range(1000):
pass
end = [Link]()
print("Time taken for the execution of code in seconds", end-start)

start = time.perf_counter()
for i in range(1000):
pass
end = time.perf_counter()
print("Time taken for the execution of code in seconds", end-start)
Explanation:
 [Link]() and time.perf_counter() are used to measure performance time.
 perf_counter() is more accurate.

22. Time calculations: last week

Example:
start = [Link]() - [Link](weeks=1)
end = [Link]()
end - start
Explanation:
 Calculates current date minus 1 week.
 Then computes time difference between end and start.

23. Converting string to datetime

Example:
from datetime import datetime
date = "2024-07-10"
date = [Link](date, "%Y-%m-%d")
print(date)
Explanation:
 strptime: Parses string into a datetime object.

24. Using Timedelta to add time

Example:
time = [Link](days = 1, hours =5, minutes=40)
dt = pd.to_datetime("2024-07-10")
dt + time
Explanation:
 Timedelta: Represents time duration.
 Adds duration to a specific date.

25. Fetching stock data using yfinance

Example:
import yfinance as yf

data = [Link]('GOOG', start = '2023-03-01', end = '2024-07-01')

data

data['Close'].resample('M').mean()
data['Close'].resample('D').mean()
Explanation:
 Downloads historical stock prices for Google.
 resample('M'): Monthly average of closing price.
 resample('D'): Daily average.

26. Plotting in pandas

Example:
d = [Link]([1, 2, 3, 7, 8, 9, 6])
[Link]()

data = {"A": [1, 2, 3], "B": [4, 5, 6]}

df = [Link](data)
[Link]()
df['A'].plot(kind = 'line')
df['A'].plot(kind = 'bar')
df['A'].plot(kind = 'hist')
[Link](x = 'A', y = 'B', kind = 'scatter')
Explanation:
 plot(): Draws basic charts.
 'line': Trend over time.
 'bar': Comparison between categories.
 'hist': Frequency of values (distribution).
 'scatter': Relationship between two variables.
 Matplotlib
1. Why Do We Need Data Visualization?
Explanation:
 Data visualization helps us understand and summarize large data in a simple, visual way.
 It makes it easier to spot patterns, trends, and outliers.
 Visuals like charts and graphs help convey complex information quickly.

2. Common Python Libraries for Visualization

Explanation:
 Matplotlib: Used for creating static, animated, and interactive plots.
 Seaborn: Built on top of matplotlib, provides prettier and more informative plots.
 Plotly: For interactive and web-based visualizations.
 Bokeh: Great for interactive visualizations in web apps.
3. Matplotlib Summary
Explanation:
 Matplotlib is a powerful Python library.
 It lets you create everything from simple charts to complex visualizations.
 Aim: "Easy things easy, hard things possible. through visulization"

4. Example: Importing Required Libraries

import pandas as pd
import numpy as np
import [Link] as plt
import warnings
[Link]('ignore')
Explanation:
 pandas and numpy: Used for data manipulation and numerical operations.
 [Link]: Module used to plot graphs.
 [Link]('ignore'): Hides warning messages in output.

5. Common Plot Types

Explanation:
 Line plot: Shows trends over time.
 Scatter plot: Shows relationship between two numeric variables.
 Bar plot: Used for categorical data.
 Histogram: Shows distribution of numerical data.
 Pie chart: Shows percentage breakdown of categories.
 Box plot: Shows distribution and outliers.
 Violin plot: Shows distribution + probability density.

6. Example: Generating Random Numbers

x = [Link](50)
Explanation:
 Generates 50 random float values between 0 and 1.
 Stored in variable x.

7. Example: Creating a DataFrame

df = [Link]([Link](1000), columns=['Data'], index=pd.date_range('2024-07-11',
periods=1000))
df
Explanation:
 Creates 1000 rows of random numbers (normally distributed).
 Adds a date index starting from July 11, 2024.
 Column name is Data.

8. Example: Plot the Data

[Link](figsize=(20, 8))
Explanation:
 Plots the Data column.
 figsize=(20, 8) sets the width and height of the plot.
 You can see how values fluctuate, mostly between -2 and 3.

9. Example: Scatter Plot

What is Scatter Plot?
A scatter plot is a graph that shows the relationship between two things using dots.
In simple words:
Each dot represents a pair of values (like height and weight).
 The dots are placed on a graph with two (x and y)— one for each value.
It helps us see patterns, like whether the two things increase or decrease together.
Example:
If you plot students’ study hours (x-axis) and their marks (y-axis), a scatter plot will show whether more
study hours lead to better marks.

x = [Link](50)
y = [Link](50)
[Link](x, y)
Explanation:
 Generates 50 random values each for x and y.
 [Link](x, y) plots them as dots on a graph.

10. Example: Line Plot with Titles and Labels

A line plot is a graph that shows how something changes over time using connected lines.
In simple words:
 It uses points to show values.
 The points are connected by lines to show the trend or pattern.
 It’s great for showing growth, decline, or fluctuations over time.
Example:
If you plot the temperature each day for a week, a line plot will show how the temperature goes up or
down day by day.

x = [1, 2, 3, 4, 5]
y = [5, 2, 7, 8, 1]
[Link](x, y)
[Link]("Line chart example")
[Link]("Day")
[Link]("Stock Price")
[Link]()
Explanation:
 x represents days, y represents stock prices.
 Adds a title, and labels for X and Y axes.
 [Link]() displays the plot, it blocks the unwanted text displaying without show().

11. Example: Customize Line Plot

[Link](x, y, color="red", marker="o", linestyle="--", linewidth=4, markersize=15)
[Link]("Styled Line Chart")
[Link]("Day Interval")
[Link]("Price")
[Link]()
[Link]()
Explanation:
 color="red": Line is red.
 marker="o": Points are shown as circles.
 linestyle="--": Dotted line.
 linewidth=4: Line is thicker.
 markersize=15: Bigger circles.
 [Link](): Shows background grid.

12. Example: Multiple Lines on One Plot

x = [1, 2, 3, 4, 5]
y1 = [5, 1, 7, 8, 2]
 y2 = [7, 8, 3, 5, 1]
[Link](x, y1, label="Tata Motors")
[Link](x, y2, label="Tata Power")
[Link]()
[Link]()
Explanation:
 Plots two lines on the same chart.
 label helps identify which line is which.
 [Link]() displays the labels written inside plot() function.

13. Example: Sine and Cosine Plots

x = [Link](0, 10, 100)
[Link](x, [Link](x))
[Link](x, [Link](x))
[Link]()
Explanation:
 [Link](0, 10, 100): 100 numbers from 0 to 10.
 Plots sine and cosine curves.

14. Example: Subplots (1 Row, 2 Columns)

If you want to show two or more sub plots in one page
[Link]() #initialize the page to make plot
[Link](1, 2, 1) #1 means one row, 2 means two columns, 1 means enter into subplot 1
[Link](x, [Link](x))
[Link](1, 2, 2) #1 means one row, 2 means two columns, 1 means enter into subplot 2
[Link](x, [Link](x))
[Link]()
Explanation:
 [Link](1, 2, 1): First of 2 plots in 1 row.
 Useful for comparing two plots side by side.
 similarly if you want to show subplots in one row and two columns then just write 2 in first field of
subplot function and 1 in second field of subplot function i.e subplot(2,1,1). here the highlighted red
colours shows that 2 rows and 1 columns

15. Example: Subplots (2 Rows, 2 Columns)

showing four subplot in one page
[Link]()
[Link](2, 2, 1); [Link](x, [Link](x)) # 2 means two rows, 2 means two columns =>this is like 2*2
matrix, 1 means first subplot
[Link](2, 2, 2); [Link](x, [Link](x)) #2 means two rows, 2 means two columns, 2 means second
subplot
[Link](2, 2, 3); [Link](x, [Link](x)) #2 means two rows, 2 means two columns, 3 means third
subplot
[Link](2, 2, 4); [Link](x, [Link](x+1)) #2 means two rows, 2 means two columns, 4 means fourth
subplot
[Link]()
Explanation:
 Creates a 2x2 grid of plots just like matrix.

16. Example: Axis and Limits

[Link](x, [Link](x))
[Link](-1.5, 1.5)
[Link](1, 15)
 [Link]('equal')
[Link]()
[Link]()
Explanation:
 [Link]() and [Link](): Sets custom Y-axis and X-axis range.
 axis('equal'): Equal scale on both axes. This helps to make sure that plot is not empty
 grid(): Adds background grid.

17. Example: Histogram

what is Histogram?
A histogram is a graphical representation of the distribution of numerical data. It uses bars to display the
frequency or number of data points within specified ranges or bins
 Histograms group numerical data into ranges called bins or intervals.
 Each bar in the histogram represents a bin, and the height of the bar indicates the frequency (number
of data points) within that bin.
In simple words:
 It groups similar values together (like 0–10, 10–20).
 Each bar shows how many times values fall in that range.
 It helps us understand the distribution of continuous numbers(decimal values).
Example:
If you have test scores of 100 students, a histogram shows how many students scored between 50–60,
60–70, and so on.
Note: sometimes bins are more but we don’t see them into the form of intervals or ranges(like 10, 20,30,
…) why because it is visible in bar form. Sometimes bins is created but not visible due to insufficient
data, look in the below graph there 10 bins are created but bins(1 and 2) is not visible due to insufficient
data.

data = [Link](1000)
[Link](data, bins=20, color='red')
[Link]()
Explanation:
 Shows distribution of data.
 bins=20: Divides into 20 intervals. by default bins are divided into 10 mins but you can explicitly
mention the bins.

18. Example: Stacked Histogram

A stacked histogram is a special type of histogram that shows multiple groups of data layered on top
of each other, in the same plot.
In Simple Words:
 Instead of plotting each dataset separately, a stacked histogram adds them together vertically.
 Each colored section of a bar represents a different group (or dataset).
 It helps you compare the overall distribution and see each group’s contribution.
 data1 = [Link](0, 1, 5000)
data2 = [Link](3, 2, 5000)
[Link]([data1, data2], bins=20, color=['r', 'b'], label=['dist1', 'dist2'])
[Link]()
[Link]()
Explanation:
 Two datasets plotted together using square brackets.
 Helpful to compare distributions.

19. Example: Bar Plot

Bar plot is used to see graph of categorical data
x = ["a", "b", "c", "d"]
y = [4, 6, 1, 10]
[Link](x, y, color='red') # shows vertical bars with categories on y-axis
[Link](x, y, color='red') # shows horizontal bars with categories on x-axis
[Link]()
Explanation:
 Shows values for categories (like students).
 Vertical and horizontal formats.

20. Example: Pie Chart

langs = ["C", "Python", "Java"]
students = [20, 100, 40]
[Link](students, labels=langs, autopct="%1.1f%%")
[Link]("Pie Chart")
[Link]()
Explanation:
 Displays percentage share of each language.

21. Example: Focus on One Segment in Pie Chart, use explode as a parameter in pie function
explode = (0.0, 0.1, 0.1)
[Link](students, labels=langs, autopct="%1.1f%%", explode=explode, shadow=True)
[Link]("Exploded Pie Chart")
[Link]()
Explanation:
 explode: Separates selected slices.
 shadow=True: Adds shadow effect.

22. Example: 3D Scatter Plot

fig = [Link]()
ax = fig.add_subplot(projection='3d')
[Link](x, y, z, c='red')
Explanation:
 Plots in 3D space with x, y, z axes.
 Requires projection='3d'.

23. Example: Plot Real Dataset

data = pd.read_csv("Bank_Churn.csv") #dataset of customer exited from bank
[Link](x=data['Age'], y=data['Balance'])
[Link](data['Age'], color='red')
[Link](data['Balance'], color='red')
[Link](column='Balance') #boxplot helps to identify the outliers
 [Link](data['Balance'])
Explanation:
 Uses a real dataset (Bank_Churn.csv).
 Shows scatter, histograms, boxplot, violin plot to explore data.
 Helps understand distribution and outliers.

24. Example: Area Plot

years = [2021, 2018, 2022, 2023, 2017]
category1 = [20, 25, 30, 28, 35]
category2 = [10, 15, 20, 18, 25]
category3 = [5, 8, 10, 9, 15]

[Link](years, category1, category2, category3,

labels=["category1", "category2", "category3"],
colors=["skyblue", "lightgreen", "orange"])
[Link]("Years")
[Link]("Values")
[Link]("Area Plot")
[Link]()
[Link]()
Explanation:
 Area plots show changes in values over time.
 Stacks categories to show overall contribution.
 Good for showing how parts contribute to a whole.