Scribd
Upload
2 views7 pages

Data Preprocessing

The document outlines various forms of data preprocessing, including data cleaning, integration, reduction, transformation, and discretization. Key techniques discussed include handling missing and noisy data, entity identification, dimensionality reduction, and normalization strategies. It emphasizes the importance of tools and methods for effective data management and preparation for analysis.

Uploaded by

veeyes16
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views7 pages

Data Preprocessing

The document outlines various forms of data preprocessing, including data cleaning, integration, reduction, transformation, and discretization. Key techniques discussed include handling missing and noisy data, entity identification, dimensionality reduction, and normalization strategies. It emphasizes the importance of tools and methods for effective data management and preparation for analysis.

Uploaded by

veeyes16
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Data Preprocessing

Forms of Data Preprocessing


Data Cleaning

1. Missing Data 2. Noisy Data 3. Data Cleaning as a process


 Ignore the tuple  Binning  discrepancy detection
 Fill in the missing value  Field overloading
manually  unique rule
 Use a global constant  consecutive rule
to fill in the missing  null rule
value  Data scrubbing tools
 Use a measure of  Data auditing tools
central tendency for  Data migration tools
the attribute  ETL
 Use the attribute (extraction/transforma
mean or median for all tion/loading) tools
samples belonging to
the same class as the  Regression
given tuple  Outlier Analysis by
 Use the most probable Clustering
value to fill in the
missing value
Data Integration

1. Entity Identification Problem


2. Redundancy and Correlation
Analysis
 Chi-square
 correlation coefficient
 covariance
3. Tuple Duplication
4. Data Value Conflict Detection
and Resolution
Data Reduction

1. Dimensionality reduction
 Wavelet transforms
 principal components
analysis
2. Attribute subset selection
3. Numerosity reduction
 parametric methods
 Regression and log-
linear models
 nonparametric methods
 histograms
 clustering
 sampling
 data cube aggregation
4. data compression
 lossless
 lossy
5. Data Cube Aggregation
Data Transformation and Data Discretization

1. Data Transformation Strategies Overview


 Smoothing
 Attribute construction
 Aggregation
 Normalization
 Min-max normalization
 z-score normalization
 Discretization
 Binning
 Histogram Analysis
 Cluster, Decision tree and Correlation
analysis
 Concept hierarchy generation for nominal
data

You might also like