Python for data science and data analysis
Data Analysis
Data analytics is important because it helps businesses optimize
their performances. Implementing it into the business model
means companies can help reduce costs by identifying more
efficient ways of doing business and by storing large amounts of
data.
Data Science
Data science is the study of data to extract meaningful
insights for business. It is a multidisciplinary approach
that combines principles and practices from the fields of
mathematics, statistics, artificial intelligence, and
computer engineering to analyze large amounts of data.
data analysis => Descriptive
data science => predictive
Python code :
1. Pycharm
2. Jupyter(IDE + ONLINE IDE)
3. Google colap
Install : jupyter:
Print (“abir das”) – string
Print (20) – number
Data Type:
error & bug create => data type
1. None - a => variable but no value assign
2. Numeric =>
a. Int – 1,10,100
b. Float – 1.1,3.1
c. Boolean- true ,false
d. Complex- a+bi;
3. List –
4. Tuple
5. Set
6. String
7. Range
8. Dictionary => mapping – huge data - fetch – fastly =>
every key assign a value
Python Operator:
1. Arithmetic => +,-,*,/,%
2. Assignment
3. Unary
4. Logical :
- And : condition both true => statement => print
- Or : one condition true => statement => print
- Not : input -> opposite => 0 -> 1
And :
x/a y/b z/c
0 => false 0 =>false false
0 1 => true 0
1 0 0
1 1 1
Or :
x/a y/b z/c
0 => false 0 =>false false
0 1 => true 1
1 0 1
1 1 1
5. Conditional
Number system conversion:
Hexa = base 16(0-9,10-15=> A -f)
Decimal = base 10(0-9)
Octal = base 8(0-7)
Binary = base 2(0,1)
Decimal to binary:
10 ->2 =
12 ->
Deep learning: all about neural network =>
1. Single layer perceptron
2. multi-layer perceptron
Framework: a. TensorFlow b. Pytorch
Machine learning:
1. Supervised learning: many algorithms -> data are
labeled => you have to find input from output (feature
and label)
2. Unsupervised learning: every data has own character
and we have to find relationship between them(group
then clustering)
Artificial Intelligence:
1. DL 2. NN 3. ML 4. NLP 5. VISION ROBOTICS
FUZY LOGIC
UNSUPERVISED ALGO SUPERVISED ALGO
CLUSTERING ALGO REGRESSION-
AND DIMENSIONALITY 1. LINEAR
REDUCTION 2. POLYNIOMIAL
SVD DECISION TREE
PCA RANDOM FEROST
K-MEANS
1. APRIORI 1. KNN
2. FP GROWTH 2. TREES
3. LOGISTIC REGRESSION
4. NAÏVE BAYES
5. SVM
Linear Regression
Scikit-Learn, also known as sklearn is a python library to
implement machine learning models and statistical
modelling. Through scikit-learn, we can implement
various machine learning models for regression,
classification, clustering, and statistical tools for
analyzing these models.
Regression evaluation Matrices
Logistic regression
What:
1. Classification problem
(spam vs ‘ham’ email; loan default; diseases
diagnosis => binary classification )
2. Logistic regression estimates the probability of
an event occurring, such as voted or didn't vote,
based on a given dataset of independent
variables.
Why Logistic Regression?
How Logistic Regression work ?
Sigmoid is a mathematical function that takes any real
number and maps it to a probability between 1 and 0.
How ?
S1. Take a linear regression solution
S2. Put into sigmoid function
S3. Transform linear model into logistic model
0.5 => cut off =>
Less than 0.5 => class 0 ; greater than 0.5 => class 1 =>
binary classification
S4. After train logistic model on data => evaluate model
performance on test data => use confusion matrix
Confusion matrix
N=165 Predicted (No) Predicted(Yes)
Actual(No) 50 10
Actual(Yes) 5 100
No = negative => test = false = 0
Yes = positive => test = True = 1
Classfier predict yes => 110
N0 => 55
Realtity yes => 105
No => 60
True positive =predict yes->Actual yes = 100
True negative = predict no -> Actual no=50
False positive = predict yes -> Actual no = 10
False negative = predict no -> Actual yes=5
How often it is correct?
Correctness = true positive + true negative/ total
= 100 + 50 / 165 = 0.91
Error = False positive + false negative/ total
= 10 + 5 / 165 = 0.09
Decision Tree and Random Forest
Decision tree
Random forest
Random forest = more than one decision tree /many
Weakness : predictive analysis is not possible
K – Means Clustering