-
Chaptero : Introduction - Step in KDD
) Definial Automatically discover useful information in large data
:
repositories
- KDD(knowledge Discovery in Databases) : Nontrivial process of
identifying valid,
novel , potential usage and
1) Data Preparation Select clean
:
, ,
understandable patterns in
↓ Integration data
2) Data mining :
Apply algo to enumerate pattern
↓
3) Pattern Evaluation :
Define subset of patterns that contain knowledge
② problems :
Scalability for high dimensional data handling noise , , privacy
preserving
-Chapter 1 : Data
preprocessing
and Attributes
7
Data
Types :
-
Data objects Samples :
,
instances , points - Entities in database
-
Attribute
types Nominal :
categorical State (Name I order)
,
Binary Sym/Asym (Oand1-Importance)
:
Original
meaningful order unknown magnitude
:
,
Numeric : interval-scaled/ratio-scaled
1)
Data
quality and preprocessing tasks :
Loc absoluteo
Realistic?
-
Quality :
Accuracy :
completeness Missing : attributes ?
consistency
Timeliness
Believability Integration
-
preprocessing tasks :
1)cleaning IXlj althies loc nhirvabo outliers
:
,
3) Transformation Min-max/2-score :
Reduction PCA
Dimensionality
3) :
>
MUST , because : 1) Tremendous amount of data (need Reduction)
2) High dimensionality
3) Data Complexity (need Integration)
- Chapter 2 Classification
:
whiem vucia data mining ,
4) supervised/Unsupervised Learning
:
-
supervised (Classification) Learning Training data accompanied by labels :
-
Unsupervised (Clustering) Learning Unknown class labels :
Goal Establish possible clusters :
) classification algorithms
:
+
1) Dicision Tree Induction Top-down recursive divide-and-conquer
:
,
v Attribute selection Information
- gain select highest entropy reduction
:
,
Stopping condition All sample same class/a remaining attributes
·
:
2) Bayes method :
Naive Bayes
-
Assume conditional independence
:
- Laplacian Correction : Avoid
zero-probability (Add 1 to each count)
Learners (k-nearest neighbor
3) Lazy
Store training examples and delay processing until test instance is
classified
>
Selection of K :
Small 1 :
overfitting
Large : irrelevant points
#
model evaluation and selection :
-
Confusion matrix : Tableto
analyze classifier performance
(TP + TN) (Tre/All)
-
Metrics :
1) Accuracy :
All
TP)TP + NF)
2) Precision : % correctly identifiedpos/s TP/(TP + FR)
f
3) Recall (sensitivity)
correctly identified pos/All red pos : %
2 PAR
4) El-measure Harmonic mean ofPrecision and Recall F
: :
,
=
P R +
validation method 1) Holdout
-
Randomly partition datato training (2/3)
: :
and test (1/3)
Cross-validation
2) : k exclusive subsets (ntest and
-
ROC curves : Trade-off TP and FP , AUC measures
accuracy K-I train)
*
Decision Tree induction :
Top-down recursive divide-and-conquer
, ,
-
Information Gain :
1) Entropy of dataset (Info(D)) :
>
uncertainty -
3 AK
ding :
2)
1) Cingnhan
Het attri
INFOLD) =
- Pigapi
3) Het data Pi : % mai eC
2) Expected into after splitting by attribute A :
InFoALD) = xInfo(D
3) calculate into gain :
Gain (A) =
Info(D) InFoA(D) -
-
2 classes : Yes/No
1 ↓ 5
I
E
2 - 4 attributes :
2
-
14 data objects :
b
1
1
① INFOLD) =
- (92(p) + 192()
L
A 2 ,94
0
② For (1) :
5R : 3N2y - 1(2 3) = 0,971
2 ,
ER WH E gY I 55
40
:
:
2N3y
44 >
-
>
- 1(3 2)
[(4, 0)
,
=
=
0
,971
0
(Leaf)
5N
40
(1)
GM
(2) (3) (4)
Infoci (D) =
=p .
1(2 3)
, +
=p .
1(3 2)
, +
124 0)
,
= 0 694
,
③ G((l)) = 094 -0. 654
.
= 0. 246
For (2) : 4H ! 2Y 2N : 1(2 2) = 1
&
,
4C : 34 1N : I(3 1)
,
= 0.918 Infoc(D) = 1(2 2) + · 1(3 1) + 2 [14 2) , ,
.
,
GM 14
: 4Y 2N :
1(4 2)
,
= 0,811
= .911
0 -
>
G((2)) = 0,029
INFO(s) (D)
: EH3YN T 1 1(3 4) 1 16,
For (3) = .
,
+ .
G((3)) =
For (4)
:SF:GNS
Infoc) (D) =p ·
116 , 2) +
↑ 1(4 2) ,
G((P))
Outlook
/17
R O S
?
Humid)(yes) /Windy
High Normal True False
(NO) (yes) (NO) (yes]
2)
Chorboundary saw cho 12 -> GAGR
↑ a
· enropytrain
Info(D) ( 192(2) 19c())
Numeric
-
(1) ② = -
+
(2)
-
1
=
=
>
② Force) :
4 : 4A OR - [(4 0)
,
= 1
-
M4 : 2A2R >
-
1(2 2)
,
= 0
W4 : OA4R -- 110 4) ,
= 1 .
Info(i) (D) = .
1
+ + 0
3
=
G((u))
=
ForL2) : PS :
OA3R >
-
I(03) = 1 .
S
.
G4 : 2A2R -> 1(2 2) ,
= 0 Info(t)(D) =
! . 1+
10 .
E5 : 4A1R-1(4 1) ,
=
=
For GPA :
Boundary : 3 4-3 5
, ,
P
313
=
.
5 =
3 45
.
GPA> 3 45 - GAOR - 116, 0) 1
G(GPA)
=
(GPA-3 45 1
.
>
- =
OAGR
.
.
>
- -
> [(06) , = +
For SAT :
Boundary : 1350-1350
GLSAT) = 1 .
> Chon GPA vi: 1) Xuthier tric
Phamvihephon (0-4) >(100-1600)
/
e)
when ?
~
E >Presence of
noise
⑦
-) Interpretability 1) Overfitting 1) Explanability
2) No feature scaling 2) Instability (High 2) Mix datatype
variance)
Bothcategorica/Numericis
3) Preliminary analysis
3) Greedy , a
reach 4) Nonlinear relationship
from Information Gain optimal
4) Hard with continuous
label
PLHIX) P(XIH) DCH)
* Bayes method
: =
P(X)