Data Science Projects Lifecycle

TDSP and CRISP Methodologies

There are two well-known Data Science methodologies for project management, namely, CRISP-DM (Cross-
Industry Standard Process for Data Mining) and Microsoft TDSP (Team Data Science Process).

The first version of CRISP-DM was proposed in 1999 as a result of a concentrated effort to identify and set out
industry guidelines for the data mining process. Since then, several refinements and extensions were
proposed. As of 2014, CRISP-DM was the most widely used methodology for analytics, data mining, and data
science projects. In October 2016, Microsoft introduced a new data science methodology called TDSP to leverage
well-known frameworks and tools such as git version control.

The aim of both these methodologies is to provide Data Science teams with a systematic approach, built on the
industry's best practices, to guide and structure Data Science projects, improve team collaboration, enhance
learning and ultimately, ensure quality and efficient results throughout the project development and delivery of
data-driven solutions.

Cross-Industry Standard Process for Data Mining (CRISP-DM)

CRISP-DM is a 6-step planning methodology, with each step comprising a sequence of events. As represented in
the image below, some of the steps are iterative, often requiring returning to previous tasks. This reflects the
non-linear data science workflow.

The six steps represented here are:

1. Business Understanding: focuses on understanding the project objectives and requirements from a
business perspective, and then translating this information into a Data Science problem definition.
2. Data Understanding: focuses on collecting and becoming familiar with the data; this is relevant to
identify data quality problems, discover first insights into the data and form hypotheses.
3. Data Preparation: aims to transform the raw data into a final dataset that can be used as input to
modelling techniques (e.g., Machine Learning algorithms).
4. Modeling: involves applying different modelling techniques to the dataset in order to generate a set
of candidate models.
5. Evaluation: once the models have been built, they need to be tested to ensure they generalise
against unseen data and that all key business objectives have been considered (e.g., the final model
needs to be fair, human-interpretable, and achieve an accuracy X% higher than the client's current
solution). The outcome of this stage is the champion model.
 6. Deployment: the champion model is deployed into production so it can be used to make predictions
on unseen data. All the data preparation steps are included so that the model will treat the new raw
data in the same way as during the model development.

Team Data Science Process (TDSP)

In October 2016, Microsoft introduced the Team Data Science Process as an Agile, iterative Data Science
methodology built on Microsoft's (and other companies) best practices, in order to facilitate the successful
implementation of Data Science projects.

The process comprises four key components:

1. Data Science Lifecycle definition

2. Standardized project structure
3. Infrastructure and resources for Data Science projects
4. Tools and utilities necessary for the project execution

Data Science Lifecycle

TDSP provides a lifecycle to structure the development of data science projects, outlining all the steps that are
usually taken when executing a project. Due to the R&D nature of Data Science
projects, employing standardized templates helps to avoid misunderstanding by enhancing the ability to
communicate tasks to other members of the team, as well as to the clients, by using a well-defined set of artefacts.

TDSP Lifecycle is made up of 5 stages:

1. Business Understanding
2. Data Acquisition & Understanding
3. Modeling
4. Deployment
5. Customer Acceptance

1. Business Understanding: this stage involves the identification of the business problem, the definition of the
business goals and the identification of the key business variables the analysis needs to predict. The metrics that
will be used to assess the success of the project are also defined in this stage. Another important step includes
surveying the available data sources and understanding the kind of data that is relevant for answering the
questions underlying the project goals. This analysis will help determine if data collection or additional data
sources will be needed.

2. Data Acquisition and Understanding: being data the key ingredient of any data science project, the second
stage revolves around data. It is essential to assess the current state of the data (how messy and unreliable is
it?), its size and quality, before moving on to the modelling stage. In this stage, the data is explored,
preprocessed and cleaned. This is essential not only to help data scientists build an initial data understanding,
but also to avoid propagating errors downstream and increase the chances of obtaining a reliable and accurate
model. This stage also aims at finding patterns in the data to guide the choice of the most appropriate modelling
techniques to use. At the end of this stage, the data scientists usually have a better idea of whether the existing
data is sufficient, if they might need to find new data sources to augment the initial dataset, or if the data is
appropriate to help answer the questions underlying the project goals.

3. Modelling: in this stage, feature engineering is performed on the cleaned dataset in order to generate a new,
improved, dataset that facilitates model training. Feature engineering usually relies on the insights obtained
from the data exploration step and on the domain expertise of the data scientist. After ensuring the dataset is
comprised of (mostly) informative features, several models are trained and evaluated, and the best one is
selected to be deployed.

4. Deployment: this stage involves deploying the data pipeline and the winner model to a production or
production-like environment. Model predictions can be made either in real-time or on a batch basis and this has
to be decided in this stage.

5. Customer acceptance: the last stage of TDSP, for which no CRISP-DM equivalent is available, is customer
acceptance. This involves two important tasks, namely: (i) system validation and (ii) project hand-off. The goal of
system validation is to confirm that the deployed model meet the client's needs and expectations, whereas the
project hand-off includes handing-off the project to the person responsible of running the system in production,
as well as delivering any project reports and documentation.