0% found this document useful (0 votes)
11 views14 pages

Module II - Data Collection and Pre-Processing

Module II covers data collection and pre-processing strategies, emphasizing the importance of systematic data gathering for reliable analysis. It classifies data collection methods into primary and secondary, detailing techniques such as surveys, observations, and experiments, along with data sources like databases and data lakes. The module also highlights the necessity of data pre-processing, including cleaning, integration, and transformation, to enhance data quality and model performance.

Uploaded by

kmfathima37
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
11 views14 pages

Module II - Data Collection and Pre-Processing

Module II covers data collection and pre-processing strategies, emphasizing the importance of systematic data gathering for reliable analysis. It classifies data collection methods into primary and secondary, detailing techniques such as surveys, observations, and experiments, along with data sources like databases and data lakes. The module also highlights the necessity of data pre-processing, including cleaning, integration, and transformation, to enhance data quality and model performance.

Uploaded by

kmfathima37
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Module II: Data Collection and

Pre-Processing
Unit 5: Data Collection Strategies
●​ Core Concept
○​ Data collection is the systematic process of gathering and measuring information on
targeted variables in an established system.
○​ It enables one to answer relevant questions and evaluate outcomes.
○​ The quality of the collected data directly impacts the quality of the analysis and the
reliability of the resulting insights.
●​ Classification of Data Collection Methods
○​ Data can be broadly classified based on its source.
○​ Primary Data: Data collected firsthand by the researcher or data scientist for a
specific purpose.
■​ It is original, raw, and collected directly from the source.
■​ Advantages: High control over data quality, tailored to specific research
needs, more reliable and authentic.
■​ Disadvantages: Time-consuming, expensive, requires significant planning
and resources.
○​ Secondary Data: Data that has already been collected by someone else for a
different purpose and is publicly or privately available.
■​ It is second-hand data.
■​ Advantages: Less expensive, quicker to access, can provide a larger
dataset than what could be collected firsthand.
■​ Disadvantages: May not be specific to the researcher's needs, data quality
is unknown, may be outdated or incomplete.
●​ Primary Data Collection Techniques
○​ 1. Surveys and Questionnaires
■​ Involve asking a series of questions to a group of individuals (respondents).
■​ Types of Surveys:
■​ Mail/Email Surveys: Low cost, but often have low response rates.
■​ Telephone Surveys: Higher response rate than mail, but can be
intrusive.
■​ Online Surveys (e.g., Google Forms, SurveyMonkey):
Cost-effective, easy to distribute, and data is collected in a structured
format automatically.
■​ In-Person Interviews (Structured or Unstructured): Highest
response rate and allows for clarification of questions, but is the most
expensive and time-consuming.
■​ Question Types:
■​ Closed-ended: Provide respondents with a fixed set of options (e.g.,
multiple choice, Likert scale). Easy to analyze quantitatively.
■​ Open-ended: Allow respondents to answer in their own words.
Provides rich qualitative data but is harder to analyze.
○​ 2. Observation
■​ Involves gathering data by observing subjects (people, events, objects) in
their natural setting.
■​ Types of Observation:
■​ Naturalistic Observation: Observing subjects without any intervention
in their natural environment.
■​ Participant Observation: The researcher becomes a part of the group
they are observing.
■​ Structured Observation: The researcher specifies beforehand what
behaviors are to be observed and how they will be measured.
■​ Example: A retail company observing how shoppers navigate a store to
optimize layout.
○​ 3. Experiments
■​ Involve manipulating an independent variable to measure its effect on a
dependent variable, while controlling for other variables.
■​ The gold standard for establishing cause-and-effect relationships.
■​ Key Components: Control group, experimental group, randomization.
■​ Example: An A/B test where a website shows two different versions of a
button (Version A and Version B) to different users to see which one gets
more clicks.
●​ Secondary Data Collection Techniques
○​ 1. Publicly Available Data Sources
■​ Government Data: Repositories like [Link] (US), [Link] (UK),
[Link] (India) provide vast amounts of data on demographics,
economics, healthcare, etc.
■​ Academic and Research Institutions: Universities and research
organizations often publish their datasets for public use.
■​ Non-Profit Organizations: Groups like the World Bank and the United
Nations provide global development data.
○​ 2. Private and Commercial Data Sources
■​ Company Databases: Internal sales data, customer records (CRM data),
web analytics logs.
■​ Data Vendors and Aggregators: Companies like Nielsen (market data),
Gartner (industry reports), and Acxiom (consumer data) sell datasets.
○​ 3. Automated Data Collection
■​ APIs (Application Programming Interfaces): A structured way to access
data from web services. Social media platforms (Twitter, Facebook), financial
services, and weather services provide APIs to programmatically fetch data.
■​ Web Scraping: The process of automatically extracting information from
websites. Used when an API is not available. Requires careful handling of
HTML structures and ethical considerations (respecting [Link] and terms
of service).
●​ Classification Table: Data Collection Strategies
Basis of Comparison Primary Data Collection Secondary Data Collection
Source Collected firsthand (Surveys, Collected by others (Govt.
Experiments, Observation) portals, APIs, Databases)
Basis of Comparison Primary Data Collection Secondary Data Collection
Cost High Low
Time Long Short
Specificity Highly specific to the project's May not be specific; collected
needs for another purpose
Control Full control over data quality No control over data quality
and collection process
Example Conducting a survey to gauge Using census data to
customer satisfaction. understand population density.
Unit 6: Data Sources
●​ Data Sources: A Deeper Dive
○​ A data source is the location where data originates. It can be a database, a file, a
sensor, a web service, etc.
●​ 1. Databases
○​ Organized collections of data, generally stored and accessed electronically.
○​ Relational Databases (SQL):
■​ Data is stored in tables with rows and columns.
■​ A predefined schema dictates the structure of the data.
■​ Examples: MySQL, PostgreSQL, Oracle, SQL Server.
■​ Accessed using Structured Query Language (SQL).
○​ Non-Relational Databases (NoSQL):
■​ Designed for unstructured and semi-structured data.
■​ Do not use a fixed schema.
■​ Types of NoSQL Databases:
■​ Document Stores (e.g., MongoDB): Store data in JSON-like
documents.
■​ Key-Value Stores (e.g., Redis): Store data as key-value pairs.
■​ Column-Family Stores (e.g., Cassandra): Store data in columns
rather than rows.
■​ Graph Databases (e.g., Neo4j): Designed to store and navigate
relationships between entities.
●​ 2. Flat Files
○​ A file containing data with no structured relationships between records.
○​ Comma-Separated Values (CSV):
■​ A text file where values are separated by commas.
■​ Simple, human-readable, and widely supported.
○​ JSON (JavaScript Object Notation):
■​ A lightweight data-interchange format.
■​ Uses human-readable text to represent data objects consisting of
attribute-value pairs.
■​ Common format for APIs.
○​ XML (eXtensible Markup Language):
■​ A markup language that defines a set of rules for encoding documents in a
format that is both human-readable and machine-readable.
■​ More verbose than JSON.
○​ Plain Text (.txt) and Log Files: Unstructured or semi-structured text data
generated by applications, servers, or networks.
●​ 3. Data Warehouses and Data Lakes
○​ Data Warehouse:
■​ A central repository of integrated data from one or more disparate sources.
■​ Stores historical data in a structured format, optimized for querying and
analysis (OLAP).
■​ The data is cleaned, transformed, and cataloged before being stored.
■​ Schema-on-write: The data structure is defined before the data is loaded.
○​ Data Lake:
■​ A centralized repository that allows you to store all your structured and
unstructured data at any scale.
■​ Stores data in its raw, native format.
■​ Schema-on-read: The data structure is applied when the data is read for
analysis.
■​ Provides more flexibility but requires more effort to govern and analyze.
●​ 4. Streaming Data Sources
○​ Data that is generated continuously by thousands of data sources.
○​ Requires real-time processing.
○​ Sources:
■​ IoT Devices: Sensors in smart homes, industrial machinery, wearable
devices.
■​ Social Media Feeds: Real-time stream of posts, comments, and likes.
■​ Financial Markets: Stock tickers and trading data.
○​ Technologies: Apache Kafka, Amazon Kinesis, Apache Flink.

Unit 7: Data Pre-processing: An Overview


●​ Definition
○​ Data pre-processing refers to the transformations applied to raw data before
feeding it into a machine learning algorithm.
○​ It is a crucial step to improve the quality of the data, and thereby, the performance
and accuracy of the resulting model.
○​ Often cited as the most time-consuming part of a data science project (often
60-80% of the time).
●​ Why is Pre-processing Necessary?
○​ Real-world data is often "dirty."
○​ Incomplete: Lacking attribute values, lacking certain attributes of interest, or
containing only aggregate data (e.g., missing values).
○​ Noisy: Containing errors, outliers, or meaningless data (e.g., a person's age is -10).
○​ Inconsistent: Containing discrepancies in codes or names (e.g., "New York" vs.
"NY"; rating "1,2,3" vs. "A,B,C").
○​ Unformatted: Data is not in a suitable format for the model (e.g., categorical text
data needs to be converted to numbers).
●​ Major Tasks in Data Pre-processing
○​ 1. Data Cleaning: Fill in missing values, smooth noisy data, identify and remove
outliers, and resolve inconsistencies.
○​ 2. Data Integration: Integrate data from multiple sources (databases, files) into a
coherent data store.
○​ 3. Data Transformation: Normalize and aggregate data. Convert data into forms
appropriate for mining.
○​ 4. Data Reduction: Obtain a reduced representation of the data set that is much
smaller in volume but produces the same (or similar) analytical results.
●​ Flowchart: The Data Pre-processing Pipeline
<!-- end list -->
[ Raw Data ]​
(Incomplete, Noisy, Inconsistent)​
|​
v​
+----------------------+​
| Data Cleaning |​
| - Handle Missing Data|​
| - Smooth Noisy Data |​
| - Handle Outliers |​
+----------------------+​
|​
v​
+----------------------+​
| Data Integration |​
| - Combine Sources |​
| - Resolve Conflicts |​
+----------------------+​
|​
v​
+----------------------+​
| Data Transformation |​
| - Normalization |​
| - Aggregation |​
| - Discretization |​
+----------------------+​
|​
v​
+----------------------+​
| Data Reduction |​
| - Dimensionality Red.|​
| - Numerosity Red. |​
+----------------------+​
|​
v​
[ Clean, Prepared Data ]​
(Ready for Modeling)​

Unit 8: Data Cleaning


●​ Goal: To "clean" the data by filling in missing values, smoothing noisy data, identifying
and removing outliers, and resolving inconsistencies.
●​ 1. Handling Missing Data
○​ Problem: Values are missing for one or more attributes.
○​ Strategies:
■​ a) Ignore the Tuple:
■​ Usually done when the class label is missing (in classification) or if the
tuple has many missing attributes.
■​ Simple, but not effective if the percentage of missing values is high, as
it reduces the dataset size.
■​ b) Fill in the Missing Value Manually:
■​ Tedious and not feasible for large datasets.
■​ c) Use a Global Constant to Fill in the Value:
■​ Replace all missing values with a constant like "Unknown" or -1.
■​ Simple, but the mining algorithm may mistakenly think the constant has
a special significance.
■​ d) Use a Measure of Central Tendency:
■​ For numerical data, replace the missing value with the attribute's mean
or median.
■​ The mean is sensitive to outliers, while the median is more robust.
■​ For categorical data, replace with the attribute's mode (most frequent
value).
■​ e) Use the Most Probable Value (Advanced):
■​ Use regression, a decision tree, or Bayesian classification to predict the
missing value based on other attributes in the record.
■​ Most sophisticated and often the most accurate method.
●​ Decision Diagram: Choosing a Missing Data Strategy
<!-- end list -->
[ Missing Data Detected ]​
|​
+--> [ Is the % of missing records small? ] --(Yes)--> [ Ignore
the Tuple ]​
|​
+--> [ Is the attribute numerical? ] --(Yes)--> +--> [ Are there
significant outliers? ] --(Yes)--> [ Impute with Median ]​
| |​
|
+-------------------------------------(No)----> [ Impute with Mean ]​
|​
+--> [ Is the attribute categorical? ] --(Yes)--> [ Impute with
Mode ]​
|​
+--> [ Is high accuracy required? ] --(Yes)--> [ Use Predictive
Model (Regression/Classifier) to Impute ]​

●​ 2. Handling Noisy Data


○​ Problem: Noise is a random error or variance in a measured variable.
○​ Strategies:
■​ a) Binning:
■​ Sort the data and partition it into a number of "bins" (or buckets).
■​ Then smooth by either:
■​ Smoothing by bin means: Replace every value in a bin with the
bin's mean.
■​ Smoothing by bin medians: Replace every value in a bin with
the bin's median.
■​ Smoothing by bin boundaries: Replace each value with the
closest boundary value (min or max).
■​ Example (for price):
■​ Sorted data: 4, 8, 15, 21, 21, 24, 25, 28, 34
■​ Partition into 3 bins:
■​ Bin 1: 4, 8, 15
■​ Bin 2: 21, 21, 24
■​ Bin 3: 25, 28, 34
■​ Smooth by bin means:
■​ Bin 1: 9, 9, 9 (mean is 9)
■​ Bin 2: 22, 22, 22 (mean is 22)
■​ Bin 3: 29, 29, 29 (mean is 29)
■​ b) Regression:
■​ Fit the data to a regression function. The function's predicted values
can be used as the "smoothed" data.
■​ c) Clustering:
■​ Group similar values together into clusters.
■​ Detect outliers as values that fall outside of any cluster.

Unit 9: Data Integration and Transformation


●​ Part A: Data Integration
○​ Goal: To combine data from multiple sources into a single, coherent dataset.
○​ Challenges:
■​ 1. Schema Integration and Entity Identification:
■​ How to match equivalent real-world entities from different sources?
■​ Example: cust_id in one database and customer_number in another
may refer to the same entity.
■​ Metadata should be used to help avoid errors.
■​ 2. Redundancy and Correlation:
■​ An attribute may be redundant if it can be "derived" from another
attribute (e.g., age can be derived from date_of_birth).
■​ Inconsistencies in attribute or dimension naming can also lead to
redundancy.
■​ Careful integration can help reduce/avoid redundancies and
inconsistencies.
■​ 3. Data Value Conflict Detection and Resolution:
■​ The same real-world entity may have conflicting attribute values in
different sources.
■​ Example: The price of a product may be in US dollars in one source
and Canadian dollars in another. Units may differ (metric vs. imperial).
■​ Must establish a "source of truth" or a transformation rule.
●​ Part B: Data Transformation
○​ Goal: To convert data into a format that is appropriate for the analysis or modeling
task.
○​ Strategies:
■​ 1. Normalization:
■​ Scaling attribute values to fall within a small, specified range.
■​ Useful when algorithms that use distance measures (like k-NN or SVM)
are involved.
■​ a) Min-Max Normalization:
■​ Scales the data to a fixed range, usually [0, 1].
■​ Formula: v' = (v - minA) / (maxA - minA) * (new_maxA -
new_minA) + new_minA
■​ Where v is the original value, minA and maxA are the min/max of
the attribute, and new_minA, new_maxA define the new range.
■​ b) Z-Score Normalization (Standardization):
■​ Transforms the data to have a mean of 0 and a standard
deviation of 1.
■​ Formula: v' = (v - meanA) / std_devA
■​ Useful when the data has outliers, as it does not have a fixed
range.
■​ c) Normalization by Decimal Scaling:
■​ Moves the decimal point of values of attribute A. The number of
decimal places to move depends on the maximum absolute
value of A.
■​ Formula: v' = v / 10^j where j is the smallest integer such that
max(|v'|) < 1.
■​ 2. Attribute/Feature Construction:
■​ Create new attributes from the given ones that can be more useful for
the task.
■​ Example: From height and weight attributes, create a
Body_Mass_Index attribute.
■​ 3. Aggregation:
■​ Summary or aggregation operations are applied to the data.
■​ Example: Aggregating daily sales data to produce monthly and yearly
sales totals.
■​ 4. Smoothing:
■​ Removing noise from data (covered in Data Cleaning).
■​ 5. Discretization:
■​ Dividing the range of a continuous attribute into intervals. (Covered in
Data Reduction).

Unit 10: Data Reduction and Discretization


●​ Goal: To obtain a reduced representation of the dataset that is much smaller in volume,
yet closely maintains the integrity of the original data.
●​ Why Reduce Data?
○​ Big data analysis can be computationally expensive and time-consuming.
○​ Data reduction can improve the efficiency of mining algorithms and the quality of
results by removing irrelevant information.
●​ Data Reduction Strategies
○​ 1. Dimensionality Reduction
■​ The process of reducing the number of random variables or attributes under
consideration.
■​ a) Feature Selection (Attribute Subset Selection):
■​ The goal is to find a minimum set of attributes such that the resulting
probability distribution of the data classes is as close as possible to the
original distribution.
■​ Methods:
■​ Stepwise Forward Selection: Start with an empty set and add
the "best" attribute at each step.
■​ Stepwise Backward Elimination: Start with the full set and
remove the "worst" attribute at each step.
■​ Combination of Forward and Backward: A combination of the
two.
■​ Decision Tree Induction: Trees can identify and remove
irrelevant attributes.
■​ b) Feature Extraction (Principal Component Analysis - PCA):
■​ Creates a new, smaller set of attributes (principal components) that are
linear combinations of the original attributes.
■​ These new attributes are orthogonal (uncorrelated) and capture the
maximum variance in the data.
■​ It effectively combines and reduces features while retaining most of the
original information.
○​ 2. Numerosity Reduction
■​ Replacing the original data volume with alternative, smaller forms of data
representation.
■​ a) Parametric Methods (Regression and Log-Linear Models):
■​ Assume the data fits some model.
■​ Store only the model parameters instead of the actual data.
■​ Example: For a linear regression model y = ax + b, we only need to
store a and b.
■​ b) Non-Parametric Methods:
■​ Histograms: Divide the data into "buckets" and store a summary
statistic (e.g., count, sum) for each bucket.
■​ Clustering: Partition data into clusters. Store the cluster
representations (e.g., centroids and diameters) instead of individual
data points.
■​ Sampling: Select a representative subset of the data.
■​ Simple Random Sampling: Each item has an equal probability
of being selected.
■​ Stratified Sampling: The data is divided into strata (e.g., age
groups), and random samples are drawn from each stratum.
●​ Data Discretization
○​ A form of numerosity reduction specifically for continuous numerical data.
○​ It divides the range of a continuous attribute into a finite number of intervals.
○​ Interval labels can then be used to replace the actual data values.
○​ Methods:
■​ Binning: An unsupervised method (already discussed in smoothing).
■​ Histogram Analysis: Unsupervised method using histogram buckets.
■​ Cluster Analysis: Can be used to group values into a finite number of
clusters (intervals).
■​ Decision-Tree Analysis: A supervised method that uses class information to
recursively partition the data. The splits in the tree define the interval
boundaries.

Practice Section: Module II


Part 1: Solved Examples
●​ Example 1: Imputing Missing Data
○​ Scenario: Given the Age data for 10 people: [25, 28, 22, 35, 41, 28, 50, ?, 33, 29].
The last value is missing.
○​ Solution:
■​ Step 1 (Choose Method): Let's use measures of central tendency.
■​ Step 2 (Calculate Mean):
■​ Sum = 25+28+22+35+41+28+50+33+29 = 291
■​ Count = 9
■​ Mean = 291 / 9 = 32.33
■​ Step 3 (Calculate Median):
■​ Sorted Data (without missing value): [22, 25, 28, 28, 29, 33, 35, 41, 50]
■​ The middle value is the 5th element, which is 29.
■​ Step 4 (Choose Imputation Value): The data has a high value (50) but no
extreme outliers. Both mean (32.33) and median (29) are reasonable. If we
want to be robust to potential outliers, the median is a safer choice. Let's
impute with the median.
■​ Result: Imputed Age = 29.
●​ Example 2: Smoothing by Bin Means
○​ Scenario: Smooth the following price data using 3 bins: [12, 15, 16, 21, 28, 29, 33,
35, 45]
○​ Solution:
■​ Step 1 (Partition into Bins):
■​ Bin 1: [12, 15, 16]
■​ Bin 2: [21, 28, 29]
■​ Bin 3: [33, 35, 45]
■​ Step 2 (Calculate Mean of Each Bin):
■​ Mean of Bin 1 = (12+15+16)/3 = 14.33
■​ Mean of Bin 2 = (21+28+29)/3 = 26
■​ Mean of Bin 3 = (33+35+45)/3 = 37.67
■​ Step 3 (Replace Values with Bin Means):
■​ Bin 1 becomes: [14.33, 14.33, 14.33]
■​ Bin 2 becomes: [26, 26, 26]
■​ Bin 3 becomes: [37.67, 37.67, 37.67]
■​ Result: Smoothed data: [14.33, 14.33, 14.33, 26, 26, 26, 37.67, 37.67, 37.67]
●​ Example 3: Min-Max Normalization
○​ Scenario: Normalize the value 300 from an attribute with a range of [100, 1000] to
a new range of [0, 1].
○​ Solution:
■​ Step 1 (Identify Variables):
■​ v = 300
■​ minA = 100, maxA = 1000
■​ new_minA = 0, new_maxA = 1
■​ Step 2 (Apply Formula):
■​ v' = (v - minA) / (maxA - minA) * (new_maxA - new_minA) + new_minA
■​ v' = (300 - 100) / (1000 - 100) * (1 - 0) + 0
■​ v' = 200 / 900 * 1 + 0
■​ v' = 0.222
■​ Result: The normalized value is 0.222.
●​ Example 4: Z-Score Normalization (Standardization)
○​ Scenario: Standardize the value 85 for an attribute where the mean is 70 and the
standard deviation is 10.
○​ Solution:
■​ Step 1 (Identify Variables):
■​ v = 85
■​ meanA = 70
■​ std_devA = 10
■​ Step 2 (Apply Formula):
■​ v' = (v - meanA) / std_devA
■​ v' = (85 - 70) / 10
■​ v' = 15 / 10
■​ v' = 1.5
■​ Result: The standardized value is 1.5. This means the value 85 is 1.5
standard deviations above the mean.

Part 2: Practice Question Sets


Set A: Short Answer Questions
1.​ What is the fundamental difference between primary and secondary data? Give an
example of each.
2.​ You need to collect real-time data from Twitter's feed. Which data collection method would
you use and why?
3.​ Explain the concept of "schema-on-read" as it applies to a Data Lake.
4.​ Why is data pre-processing considered the most time-consuming step in a data science
project?
5.​ Under what circumstances would you choose to impute missing numerical data with the
median instead of the mean?
6.​ Describe the entity identification problem in data integration.
7.​ What is the main purpose of normalization in data transformation?
8.​ Explain the difference between feature selection and feature extraction in dimensionality
reduction.
9.​ What is stratified sampling and why is it useful?
10.​How does a decision tree perform data discretization in a supervised manner?
Set B: Answers
1.​ Primary data is collected firsthand for a specific purpose (e.g., conducting a survey), while
secondary data has been collected by someone else (e.g., using government census
data). Primary data is more specific but costly, whereas secondary data is cheaper but
may not be a perfect fit.
2.​ I would use an API (Application Programming Interface). Twitter provides a streaming API
that allows for programmatic, real-time access to its data feed in a structured format,
which is far more efficient and reliable than web scraping.
3.​ "Schema-on-read" means that raw data is loaded into the Data Lake in its native format
without a predefined structure (schema). The structure is only applied when the data is
pulled out for analysis, providing great flexibility.
4.​ Real-world data is often "dirty"—incomplete, noisy, and inconsistent. A significant amount
of time is required for cleaning, integrating, transforming, and reducing this data to make it
suitable for a machine learning model, as model performance is highly dependent on data
quality.
5.​ The median is a better choice when the data contains significant outliers. The mean is
sensitive to extreme values and can be skewed, while the median represents the central
value of the sorted data and is more robust to outliers.
6.​ It is the challenge of matching records from different data sources that refer to the same
real-world entity. For example, a customer might be identified by cust_id in one table and
customer_number in another, and these need to be correctly linked.
7.​ Normalization scales numerical data to a fixed range (e.g., [0, 1]). This is important for
algorithms that use distance calculations, like k-NN, SVM, and clustering, as it prevents
attributes with larger ranges from dominating the distance metric.
8.​ Feature selection chooses a subset of the original features to use. Feature extraction
creates a new, smaller set of features by combining the original ones (e.g., PCA).
Selection keeps original features, while extraction creates new ones.
9.​ Stratified sampling involves dividing the population into homogeneous subgroups (strata)
and then taking a simple random sample from each subgroup. It is useful for ensuring that
the sample is representative of the overall population, especially when certain subgroups
are small.
10.​It uses class labels to find the best split points. It recursively partitions the continuous
attribute's range, choosing splits that result in the most information gain (i.e., the most
homogeneous child nodes with respect to the class), effectively creating intervals.

Part 3: Multiple Choice Questions (MCQs)


1.​ Conducting an A/B test on a website is an example of which data collection strategy?
○​ A) Observation
○​ B) Secondary Data Collection
○​ C) Experiment
○​ D) Survey
2.​ Which of the following is an example of unstructured data?
○​ A) A table in a SQL database
○​ B) A CSV file of sales records
○​ C) A collection of email bodies
○​ D) A JSON file with key-value pairs
3.​ The term "schema-on-write" is characteristic of a:
○​ A) Data Lake
○​ B) Data Warehouse
○​ C) Flat File
○​ D) NoSQL Database
4.​ Replacing a missing age value with the average age of all other records is an example of:
○​ A) Ignoring the tuple
○​ B) Manual filling
○​ C) Imputation by mean
○​ D) Smoothing by bin means
5.​ Which method is most robust to outliers when handling noisy data?
○​ A) Smoothing by bin means
○​ B) Smoothing by bin boundaries
○​ C) Smoothing by bin medians
○​ D) Regression
6.​ Scaling data to have a mean of 0 and a standard deviation of 1 is called:
○​ A) Min-Max Normalization
○​ B) Decimal Scaling
○​ C) Z-Score Normalization (Standardization)
○​ D) Aggregation
7.​ Combining first_name and last_name columns to create a full_name column is an
example of:
○​ A) Data Reduction
○​ B) Data Cleaning
○​ C) Attribute Construction
○​ D) Data Integration
8.​ Principal Component Analysis (PCA) is a technique for:
○​ A) Numerosity Reduction
○​ B) Dimensionality Reduction
○​ C) Data Cleaning
○​ D) Data Discretization
9.​ Which of these is a supervised method for discretization?
○​ A) Binning by equal width
○​ B) Histogram Analysis
○​ C) Decision Tree Analysis
○​ D) Binning by equal frequency
10.​A data professional finds that a city attribute is recorded as "NYC" in one table and "New
York" in another. Resolving this is part of:
○​ A) Data Reduction
○​ B) Data Transformation
○​ C) Data Integration
○​ D) Data Discretization
MCQ Answers:
1.​ C
2.​ C
3.​ B
4.​ C
5.​ C
6.​ C
7.​ C
8.​ B
9.​ C
10.​C

You might also like