Chapter 1: Data Science Methodology
2.1. Introduction to Data Science Methodology
A methodology provides a data scientist with a structured framework for designing AI projects. It
aids teams in deciding which methods, processes, and strategies to employ to achieve the
desired output efficiently. By following a systematic approach, projects can be completed without
unnecessary loss of time or [Link] Science Methodology is characterized as an iterative
process with a prescribed sequence of steps used to approach a problem and find a solution. It
specifically enables the capacity to handle and comprehend [Link] methodology discussed in
this guide was put forward by John Rollins, a Data Scientist at IBM Analytics. It consists of ten
steps organized into five modules:
1. From Problem to Approach: Business Understanding and Analytic Approach.
2. From Requirements to Collection: Data Requirements and Data Collection.
3. From Understanding to Preparation: Data Understanding and Data Preparation.
4. From Modelling to Evaluation: AI Modelling and Evaluation.
5. From Deployment to Feedback: Deployment and Feedback.
Module 1: From Problem to Approach
2.1.1. Business Understanding
This stage focuses on identifying the specific problem to be solved. Data scientists seek to
comprehend customer requirements through questioning and discussions with stakeholders.
This process is often called Problem Scoping and defining .Tools used in this stage include:
● 5W1H Problem Canvas: Used to gain a deep understanding of the issue (Who, What,
Where, When, Why, How).
● Design Thinking (DT) Framework: Employed to identify objectives that support the
customer's goal.
2.1.2. Analytic Approach
Once the business problem is established, the data scientist defines the analytical path. This
involves asking clarifying questions to determine the appropriate mathematical or logical
approach. Common questions include:
● Regression: How much or how many?
● Classification: Which category does the data belong to?
● Clustering: Can the data be grouped?
● Anomaly Detection: Is there an unusual pattern?
● Recommendation: Which option should be given to the customer?
Types of Data Analytics
Analytics Type,Focus,Purpose
Descriptive,Summarizing historical data.,"Identify patterns, trends, and anomalies in past data."
Diagnostic,Understanding why events occurred.,Uncover root causes and contributing factors.
Predictive,Predicting future outcomes based on history.,Forecast future events or behaviors.
Prescriptive,Determining the best course of action.,Recommend specific actions based on
predictive insights.
Module 2: From Requirements to Collection
2.1.3. Data Requirements
The analytic approach determines what data is needed. This stage defines the data content,
formats, and sources. Data is generally categorized into three types:
● Structured Data: Organized in tables (e.g., customer databases).
● Unstructured Data: No predefined structure (e.g., social media posts, images).
● Semi-structured Data: Partial organization (e.g., emails, XML files).
2.1.4. Data Collection
Data collection is the systematic gathering of observations or measurements.
● Primary Data Source: Raw, unprocessed data collected firsthand (e.g., surveys, IoT
sensors, interviews).
● Secondary Data Source: Reused data already stored in books, websites, or databases
(e.g., Kaggle, World Bank data, web scraping).
Module 3: From Understanding to Preparation
2.1.5. Data Understanding
In this stage, the data scientist evaluates if the collected data is representative of the problem.
Techniques used include:
● Descriptive Statistics: Univariate analysis and pairwise correlation.
● Visualization: Using tools like Histograms to assess data quality and initial insights.
2.1.6. Data Preparation
This is the most time-consuming stage. It involves transforming data into a state suitable for
modelling.
● Cleaning: Dealing with missing values, invalid data, and duplicates.
● Integration: Combining data from multiple sources.
● Feature Engineering: The process of selecting, modifying, or creating new variables
(features) from raw data to improve model performance (e.g., calculating the "Age of a
house" from the "Year built").
Module 4: From Modelling to Evaluation
2.1.7. AI Modelling
The modelling stage focuses on developing descriptive or predictive models.
● Descriptive Modeling: Focuses on summarizing characteristics and trends without
making predictions.
● Predictive Modeling: Uses historical data and statistical algorithms to forecast future
outcomes. It requires a Training Set (historical data with known outcomes) to calibrate
the model.
2.1.8. Evaluation
Evaluation assesses model performance using test data. It occurs in two phases:
1. Diagnostic Measures: Ensuring the model works as intended (e.g., using a decision
tree to evaluate output).
2. Statistical Significance Test: Verifying that the model accurately interprets data to
avoid "second-guessing."
Module 5: From Deployment to Feedback
2.1.9. Deployment
The trained AI model is made available to users. This may involve a limited rollout or a test
environment to build confidence. Deployment often requires collaboration with internal teams for
skills and technology integration.
2.1.10. Feedback
The final stage involves collecting results from the deployed model and user observations. This
iterative process allows data scientists to refine the model until it provides satisfactory results.
Model Validation Techniques
Model validation measures a model's accuracy and reliability on unseen data to prevent
overfitting and underfitting .
Train-Test Split
The dataset is divided into two subsets:
● Training Set: Used to fit the model.
● Test Set: Used to evaluate the model's predictions against expected values.
● Common Splits: 80/20, 70/30, or 67/33.
K-Fold Cross Validation
The data is split into k subsets (folds). The model is trained on k-1 folds and tested on the
remaining fold. This process repeats k times, ensuring every data point is used for both
training and testing at some stage.
Evaluation Metrics
Classification Metrics
● Confusion Matrix: A table summarizing True Positives (TP), True Negatives (TN),
False Positives (FP), and False Negatives (FN).
● Precision: Proportion of predicted positives that are truly positive.
Precision =TP/ (TP + FP)
● Recall: Proportion of actual positives correctly classified.
Recall = TP/ (TP + FN)
● F1-Score: The harmonic mean of precision and recall.
● Accuracy: (TP + TN)/Total Predictions
Regression Metrics
● MAE (Mean Absolute Error): Sum of absolute differences between predictions and
actual values.
● MSE (Mean Squared Error): Average of the squared distances between predicted and
actual values.
● RMSE (Root Mean Squared Error): The square root of MSE; preferred because it uses
the same units as the target variable.
Section 1: 30 Multiple Choice Questions (MCQs)
1. Who put forward the 10-step Data Science Methodology?
● A) John Rollins
● B) IBM Analytics Team
● C) Both A and B
● D) None of the above
● Answer: C
2. What is the primary purpose of a Methodology in Data Science?
● A) To write code faster
● B) To provide a framework for designing an AI Project
● C) To replace data scientists
● D) To store large datasets
● Answer: B
3. Which stage is also known as "Problem Scoping"?
● A) Data Collection
● B) Business Understanding
● C) AI Modelling
● D) Data Preparation
● Answer: B
4. The 5W1H method stands for:
● A) Who, What, Where, When, Why, How
● B) Which, What, Where, When, Why, How
● C) Who, What, Which, Where, Why, How
● D) None of the above
● Answer: A
5. Which type of analytics answers "Why is this happening?"
● A) Descriptive
● B) Predictive
● C) Diagnostic
● D) Prescriptive
● Answer: C
6. "Regression" is used to find:
● A) Which category data belongs to
● B) How much or how many
● C) Unusual patterns
● D) Groups in data
● Answer: B
7. Data in the form of social media posts or images is categorized as:
● A) Structured
● B) Semi-structured
● C) Unstructured
● D) Primary
● Answer: C
8. Which of these is a secondary data source?
● A) IoT Sensor Data
● B) Feedback Forms
● C) World Bank Open Data
● D) Direct Observation
● Answer: C
9. What is the most time-consuming stage in the Data Science Methodology?
● A) Data Collection
● B) Data Preparation
● C) Evaluation
● D) Deployment
● Answer: B
10.Creating a "Price per square foot" feature from "Total Price" and "Area" is an
example of:
● A) Data Cleaning
● B) Data Collection
● C) Feature Engineering
● D) Data Understanding
● Answer: C
11.Which technique is used to assess data quality and initial insights?
● A) Feature Engineering
● B) Descriptive Statistics
● C) Deployment
● D) Regression
● Answer: B
12.In AI Modelling, a "Training Set" is:
● A) Data with unknown outcomes
● B) Historical data with known outcomes
● C) Data used only for deployment
● D) Data collected from social media
● Answer: B
13.Which phase of evaluation uses "Statistical significance tests"?
● A) First Phase
● B) Second Phase
● C) Third Phase
● D) Deployment Phase
● Answer: B
14.The final stage of the methodology is:
● A) Evaluation
● B) Deployment
● C) Feedback
● D) Data Preparation
● Answer: C
15.Model Validation helps prevent:
● A) Data Collection errors
● B) Overfitting and Underfitting
● C) Business Understanding issues
● D) None of the above
● Answer: B
16.In a Train-Test split, the training set is used to:
● A) Evaluate the model
● B) Fit the model
● C) Collect feedback
● D) Clean the data
● Answer: B
17.A common train-test split percentage is:
● A) 50/50
● B) 80/20
● C) 10/90
● D) 100/0
● Answer: B
18.K-Fold Cross Validation splits data into:
● A) Two equal halves
● B) Multiple parts or folds
● C) Primary and Secondary sets
● D) Structured and Unstructured sets
● Answer: B
19.If k=5 in K-Fold validation, each fold is what percentage of the full dataset?
● A) 10%
● B) 20%
● C) 25%
● D) 50%
● Answer: B
20.Which metric is used for Regression problems?
● A) Confusion Matrix
● B) F1-score
● C) MSE
● D) Precision
● Answer: C
21.A Confusion Matrix for binary classification is of what size?
● A) 1x1
● B) 2x2
● C) 3x3
● D) NxN
● Answer: B
22.What does TP stand for in a Confusion Matrix?
● A) Total Positives
● B) True Positives
● C) Time Period
● D) Testing Phase
● Answer: B
23.Precision is calculated as:
● A) TP / (TP + FN)
● B) TP / (TP + FP)
● C) (TP + TN) / Total
● D) TN / (TN + FP)
● Answer: B
24.A perfect F1 score is:
● A) 0
● B) 0.5
● C) 1
● D) 100
● Answer: C
25.Which metric is often preferred because it is in the same units as the target
variable?
● A) MSE
● B) RMSE
● C) MAE
● D) R-squared
● Answer: B
26.Which Python library is used for train-test split and linear regression?
● A) Pandas
● B) Matplotlib
● C) Sklearn
● D) Seaborn
● Answer: C
27.Structured data is typically organized in:
● A) Tables
● B) Images
● C) Audio files
● D) Social media feeds
● Answer: A
28.In K-Fold Cross Validation, how much of the data is eventually used as a holdout
set?
● A) 20%
● B) 50%
● C) 80%
● D) 100%
● Answer: D
29.Which approach helps identify "Unusual patterns"?
● A) Clustering
● B) Classification
● C) Anomaly detection
● D) Regression
● Answer: C
30.Feedback from users helps to:
● A) Collect more primary data
● B) Refine the model and assess impact
● C) Define the business problem
● D) Choose a split percentage
● Answer: B
Section 2: 20 Short-Answer Questions
1. What is a Data Science Methodology? It is a prescribed sequence of iterative steps
that data scientists follow to approach a problem and find a solution. It provides a
framework for designing AI projects systematically.
2. Define Business Understanding. This is the first stage where the team understands
the customer's problem by asking questions and identifying objectives to support the
customer's goal.
3. What is the difference between Predictive and Prescriptive analytics? Predictive
analytics forecasts future outcomes based on historical patterns, while prescriptive
analytics recommends the best course of action to achieve a desired outcome.
4. How does Descriptive analytics differ from Diagnostic analytics? Descriptive
analytics summarizes past data to understand what happened, whereas diagnostic
analytics examines past data to understand why it happened.
5. What is Problem Scoping? Problem scoping is the process of defining the customer’s
requirements and goals clearly using frameworks like the 5W1H Problem Canvas.
6. Name the three types of data categories. The three categories are structured data
(organized in tables), unstructured data (no predefined structure like images), and
semi-structured data (like emails).
7. What is a Primary Data Source? A primary data source is the original source of raw,
unprocessed data collected firsthand through direct observation, experiments, or
surveys.
8. What occurs during the Data Understanding stage? Data scientists evaluate the
relevance and comprehensiveness of collected data using techniques like descriptive
statistics and visualization.
9. Define Feature Engineering. It is the process of selecting, modifying, or creating new
variables from raw data to improve the performance of machine learning models.
10. What is the purpose of Data Preparation? The purpose is to transform raw data into a
clean, usable state by handling missing values, removing duplicates, and integrating
multiple sources.
11. Explain Descriptive Modeling. Descriptive modeling focuses on summarizing and
understanding characteristics and patterns within a dataset without making future
predictions.
12. What is a Training Set? A training set is a portion of historical data with known
outcomes used to fit and calibrate a machine learning model.
13. Why is Model Evaluation important? It ensures the model is reliable and effective by
measuring its performance against test data before it is deployed in real-world situations.
14. What is Deployment? Deployment is the stage where a trained AI model is integrated
into a live environment and made available to users.
15. What is the role of Feedback in the methodology? Feedback from users and
deployment results is used to iteratively refine and assess the model’s performance and
impact.
16. What is a Train-Test Split? It is a validation technique where a dataset is divided into a
training subset to fit the model and a test subset to evaluate its performance.
17. Define K-Fold Cross Validation. This technique splits data into k folds, training the
model on k-1 folds and testing it on the remaining fold, repeating the process until all
data is used.
18. What is a Confusion Matrix? It is an N x N table used to evaluate classification models
by comparing predicted results against actual outcomes.
19. Define Precision in classification. Precision measures the proportion of predicted
positives that are truly positive, calculated as TP divided by the sum of TP and FP.
20. What is Mean Squared Error (MSE)? MSE is a regression metric that calculates the
average of the squared differences between predicted values and actual target values.
Section 3: 15 Long-Answer Questions
1. Describe the five modules of the Data Science Methodology by John Rollins. The
methodology is organized into five iterative modules. Module 1, "From Problem to
Approach," focuses on defining the business problem and selecting the analytical path.
Module 2, "From Requirements to Collection," identifies the data needed and the
sources to gather it from. Module 3, "From Understanding to Preparation," involves
assessing data quality and cleaning it for use. Module 4, "From Modelling to Evaluation,"
centers on building AI models and assessing their performance. Finally, Module 5, "From
Deployment to Feedback," involves releasing the model to users and refining it based on
real-world results.
2. Explain the four types of data analytics with examples. The four types are
Descriptive (summarizes past data, e.g., average student marks), Diagnostic (identifies
reasons for past events, e.g., finding why sales dropped), Predictive (forecasts future
trends, e.g., predicting future inventory needs), and Prescriptive (recommends actions,
e.g., suggesting a pricing strategy for a festival season).
3. Compare and contrast Structured, Unstructured, and Semi-structured data.
Structured data is highly organized, usually in tables or databases, making it easy to
search. Unstructured data, like images and social media posts, has no predefined format
and is more complex to process. Semi-structured data falls in between, having some
organizational elements but lacking a rigid schema, such as emails or XML files.
4. Detail the processes involved in the Data Preparation stage. Data Preparation is the
most time-consuming stage. It involves cleaning data by fixing missing or invalid values
and removing duplicates. It also includes data integration, where information from
various sources like tables and archives is combined. Finally, it involves transformation
and feature engineering, where raw data is converted into meaningful input variables for
the model.
5. Discuss the importance of Feature Engineering in model performance. Feature
engineering allows data scientists to create more relevant inputs that might not exist in
the raw data. For example, if predicting house prices, the "Year Built" is raw data, but the
"Age of the house" is a derived feature that provides a clearer signal to the model,
leading to more accurate predictions.
6. Explain the difference between Descriptive Modeling and Predictive Modeling.
Descriptive modeling is used to understand the current behavior and characteristics of
data without making decisions or forecasts. Predictive modeling, however, uses historical
data and statistical algorithms to identify trends and forecast future outcomes, such as
exam scores or stock prices.
7. Describe the two main phases of Model Evaluation. The first phase involves
diagnostic measures to ensure the model works as designed; for predictive models, this
might involve a decision tree to check alignment with the initial design. The second
phase involves statistical significance tests to verify that the model accurately interprets
data and to avoid unnecessary second-guessing.
8. What is Deployment, and why does it often require additional internal teams?
Deployment is making the model available for real-world use. It frequently requires other
teams because integrating a model into a live business process involves specialized
skills in technology, security, and infrastructure that go beyond the data scientist's
primary modelling duties.
9. Elaborate on the iterative nature of the Feedback stage. Feedback is the final but
continuous stage. Observations from how the model works in the deployed environment
and user critiques are collected to refine the model. This process continues iteratively
from modelling to feedback until the results are satisfactory and acceptable to
stakeholders.
10. Explain the Train-Test Split technique and how to configure it. This technique
evaluates algorithm performance by splitting a dataset into a training set (to fit the
model) and a test set (to evaluate it). Configuration involves choosing a split percentage,
such as 80% for training and 20% for testing. The choice depends on computational
costs and the need for the subsets to represent the overall data accurately.
11. Compare Train-Test Split with K-Fold Cross Validation. Train-Test Split is typically
applied to large datasets and has a clear demarcation between training and testing data.
K-Fold Cross Validation is better for small datasets; it divides data into folds and ensures
every data point is used for both training and testing at different stages, providing a more
reliable measure of model quality.
12. Describe the components of a Confusion Matrix for binary classification. A binary
confusion matrix has four components: True Positives (model correctly predicts 'Yes'),
True Negatives (model correctly predicts 'No'), False Positives (model incorrectly
predicts 'Yes'), and False Negatives (model incorrectly predicts 'No'). It summarizes how
well the classification model is performing.
13. Define and explain the relationship between Precision, Recall, and the F1-Score.
Precision measures accuracy in positive predictions, while Recall measures the ability to
find all actual positives. The F1-score is the harmonic mean of both; a high F1-score
indicates that the model has low false positives and low false negatives, making it a
balanced measure of performance.
14. What are the common metrics for evaluating Regression models? The common
metrics are MAE (Mean Absolute Error), which measures the sum of absolute
differences; MSE (Mean Squared Error), which averages squared differences to highlight
larger errors; and RMSE (Root Mean Squared Error), which is the square root of MSE
and is preferred for its interpretability in the original target units.
15. Discuss the benefits of Model Validation. Model validation is crucial for ensuring
accurate predictions. Its benefits include enhancing the overall quality of the model,
reducing the risk of errors in live applications, and preventing the model from either
overfitting (memorizing data) or underfitting (failing to learn patterns).
Glossary of Key Terms
● 5W1H: A questioning method (Who, What, Where, When, Why, How) used for problem
scoping and identifying data requirements.
● Accuracy: The ratio of correct predictions to the total number of predictions.
● Analytic Approach: The stage in the methodology where the type of problem (e.g.,
classification, regression) is identified.
● Anomaly Detection: An analytic approach used to identify unusual patterns in data.
● Clustering: An analytic approach used to group similar data points together.
● Confusion Matrix: A table used to evaluate classification performance by summarizing
True/False Positives and Negatives.
● Data Preparation: The process of cleaning, integrating, and transforming data for
modelling.
● Data Science Methodology: A systematic, iterative process for solving AI problems,
consisting of 10 steps.
● Deployment: The process of making a trained model available for real-world use.
● Descriptive Analytics: Analysis focused on summarizing historical data and trends.
● Diagnostic Analytics: Analysis focused on uncovering the root causes of past events.
● F1-Score: A classification metric that balances precision and recall.
● Feature Engineering: Creating new features from raw data to improve model accuracy.
● K-Fold Cross Validation: A validation technique that uses multiple subsets (folds) of
data for training and testing.
● MAE (Mean Absolute Error): A regression metric measuring the average absolute
difference between predicted and actual values.
● MSE (Mean Squared Error): A regression metric measuring the average of squared
differences between predicted and actual values.
● Overfitting: A situation where a model performs well on training data but poorly on
unseen data.
● Predictive Analytics: Analysis focused on forecasting future outcomes using historical
data.
● Prescriptive Analytics: Analysis focused on recommending the best course of action.
● Primary Data: Raw data collected firsthand for a specific project.
● RMSE (Root Mean Squared Error): The square root of MSE, used for regression
evaluation.
● Secondary Data: Pre-existing data collected by others and reused for analysis.
● Structured Data: Data organized in a predefined format, such as a table.
● Training Set: The portion of data used to "teach" or fit the model.
● Train-Test Split: Dividing a dataset into two parts for fitting and evaluating a model.
● Underfitting: A situation where a model is too simple to capture the underlying patterns
in the data.
● Unstructured Data: Data with no predefined structure, such as video, audio, or images.
● Visualization: The use of charts and graphs (e.g., histograms) to understand data.