Data Sampling
Instructor: Dr. Qurat-ul-Ain
DS110 – Introduction to Data Science
Data Science as a Process
Question/ProblemFormulation
• What do we want to know?
• What problem are we trying to solve?
• What are the hypothesis we want to test?
• What are our metrics of success? 2
Data Science as a Process
Data Acquisition
• What data do we have & what
do we need?
• How will we sample more data?
• Is our data representative of the
population we want to study? 3
Objectives
✓ Census and Surveys
✓ Data Sampling
✓ Probability Samples
5
A Census
❖A census is “a complete count or survey of a
population”
❖ People living in a particular country
❖ Bacteria in one person’s gut
❖ Trees of a certain species
❖ Bolts on a bridge
❖ Small businesses that received a microloan
❖ Published statistics in a journal (example)
6
Pakistan “Population” and Housing Census
❖ In 2023 by the Pakistan Bureau of Statistics (PBS)
❖ Counts every person living residing in Pakistan, including
nationals and non-nationals, across all provinces and
territories.
❖ The census data are used (or intended to be used) for seats
allocation, resource distribution, and delimitation.
○ Informing planning, policy, and socio-economic decision
making
In general: a census is “an official count or survey of a
population, typically recording various details of individuals.” 7
Surveys
❖A survey is a set of questions or measurements.
○ For instance: workers survey individuals and
households.
❖ What is asked, and how it is asked, can affect:
○ How the respondent answers.
○ Whether the respondent answers.
8
Sampling from a finite population Inference (Prediction):
drawing conclusions
about a population
A census is great, but expensive and difficult based on a sample.
to execute.
❖A sample is a subset of the population.
○ Samples are often used to make inferences about the
population.
○ How you draw the sample will affect your accuracy.
○ Two common sources of error:
■ chance error: random samples can vary from what is
expected, in any direction.
■ bias: a systematic error in one direction. 9
Population, samples, and sampling frame
❖ Population: The group that you want
to learn something about.
❖ Sampling Frame: The list from
which the sample is drawn.
○ If you’re sampling people, the sampling
frame is the set of all people that could
possibly end up in your sample.
❖ Sample: Who you actually end up Note: There may be individuals in your
sampling. sampling frame (and hence, your sample)
that are not in your population!
○ A subset of your sampling frame.
Diagram taken from [Link]
Single Use Case: Disease Risk Prediction Using Hospital Data
Imagine a data science project to predict the risk of developing diabetes.
Target Population
All individuals in the community who could develop diabetes
This includes:
•Healthy individuals
•Undiagnosed cases
•Mild and severe cases
•Urban and rural populations
Sampling Frame
Patients whose data exist in a tertiary hospital’s electronic health record (EHR)
This is where the data scientist actually gets data from.
Sample
A subset of hospital patients selected to train the prediction model
Single Use Case: Disease Risk Prediction Using Hospital Data
Imagine a data science project to predict the risk of developing diabetes.
Target Population There may be
All individuals in the community who could develop diabetes
individuals in your
This includes:
•Healthy individuals
sampling frame
•Undiagnosed cases (and hence, your
•Mild and severe cases sample) that are not
•Urban and rural populations in your population.
Sampling Frame
Patients whose data exist in a tertiary hospital’s electronic health record (EHR)
This is where the data scientist actually gets data from.
Sample
A subset of hospital patients selected to train the prediction model
Single Use Case: Disease Risk Prediction Using Hospital Data
Imagine a data science project to predict the risk of developing diabetes.
Target Population
There might be
All individuals in the community who could develop diabetes
This includes: individuals in your
•Healthy individuals target population
•Undiagnosed cases that are not in your
•Mild and severe cases sampling frame.
•Urban and rural populations
Sampling Frame
Patients whose data exist in a tertiary hospital’s electronic health record (EHR)
This is where the data scientist actually gets data from.
Sample
A subset of hospital patients selected to train the prediction model
Target Population
Sampling Frame
Target Population
Sample
Sampling Frame
Sample
Decreasing Bias
Target Population
Sampling Frame
Target Population
Sample Sampling Frame
Sample
Decreasing Chance Error
Common Biases
❖ Selection Bias
○ Systematically excluding (or favoring) particular groups.
○ How to avoid: Examine the sampling frame and the method of
sampling.
❖ Response Bias
○ People don’t always respond truthfully.
○ How to avoid: Examine the nature of questions and the method of
surveying.
❖ Non-response Bias
○ People don’t always respond.
○ How to avoid: Keep your surveys short, and be persistent.
○ People who don’t respond aren’t like the people who do! 16
Quality, not quantity!
❖ Tryto ensure that the sample is representative of the
population.
○ Don’t just try to get a big sample.
○ If your method of sampling is bad, and your sample is
big, you will have a big, bad sample!
Big Bad Wolf
17
Probability sampling
❖ Why?One reason is to reduce bias, but that’s not the
main reason!
○ Random samples can produce biased estimates of
population characteristics.
■ For example, if we’re estimating the maximum of a
population.
❖ Butwith random samples we are able to estimate the
bias and chance error.
○ We can quantify the uncertainty.
20
Probability sampling
In order for a sample to be a probability sample:
❖ You must be able to provide the chance that any specified
set of individuals will be in the sample.
❖ All individuals in the population do not need to have the
same chance of being selected.
❖ You will still be able to measure the errors, because you
know all the probabilities.
21
Some random sampling schemes
❖Arandom sample with replacement is a sample drawn
uniformly at random with replacement.
○ Random doesn’t always mean “uniformly at random,” but
in this specific context, it does.
❖A simple random sample is a sample drawn uniformly at
random without replacement.
○ Every individual has the same chance of being selected.
○ Every pair has the same chance as every other pair.
○ Every triple has the same chance as every other triple.
○ And so on
22
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
Is this a probability sample?
23
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
Is this a probability sample?
❖ Yes. If my sample is [n, n + 10, n + 20, …, n + 1190], where 0 <= n <= 10, the
probability of that sample is 1/10.
24
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
❖ Does each student have the same probability of
being selected?
25
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
❖ Does each student have the same probability of
being selected?
❖ Yes. Each student is chosen with probability 1/10.
26
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
❖ Is this a simple random sample?
27
Example scenario
Consider the following sampling scheme:
▪ Suppose a class roster has 1200 students listed alphabetically.
▪ Pick one of the first 10 students on the list at random.
▪ To create your sample, take that student and every 10th
student listed after that (e.g. Students 8, 18, 28, etc).
❖ Is this a simple random sample?
❖ No. The chance of selecting (8, 18) is 1/10; the chance of selecting (8, 9) is 0.
28
Thanks!
Any comment/questions?
29