CRISP-DM Guide

Summary

Presentation
The discovery of knowledge
THE CRISP-DM
1. BUSINESS/PROBLEM UNDERSTANDING.............................................................................................................................8

1.1 Determining business objectives (problem definition)..............................................................................................9

1.2 Assessing the situation
1.3 Determining the objectives of data mining
1.4 Producing a project plan
2. UNDERSTANDING THE DATA.......................................................................................................................................................15

2.1 Collecting initial data

2.2 Describing the data
2.3 Exploring the data
2.4 Verifying the quality of the data
3. DATA PREPARATION..........................................................................................................................................................20

3.1 Selecting the data

3.2 Cleaning the data
3.3 Building new data
3.4 Integrating the data
3.5 Formatting the data
4. MODELING

4.1 Selecting the modeling technique

4.2 Designing the test design
4.3 Building the model(s)...........................................................................................................................................................29
4.4 Evaluating the model(s)...............................................................................................................................................................30

5. EVALUATION

5.1 Evaluating the results..........................................................................................................................................................32

5.2 Reviewing the processes..................................................................................................................................................................33
5.3 Determining the next steps
6. IMPLEMENTATION

6.1 Planning the implementation

6.2 Planning the monitoring and maintenance
6.3 Producing the final report
6.4 Reviewing the project
RESULT IN ACTION!

Possible problems when using CRISP-DM............................................................41

Project report structure.....................................................................................................................................................43
Presentation

In order to methodologically equip beginners in Data Science,

we provide this document, a reformulation of the Cross Industry Standard Process for
Data Mining (CRISP-DM) produced in 2001 by the DaimlerChrysler/NCR/SPSS consortium, and
adapted by IBM in 2011.
This guide presents: (i) the rationale behind the creation of CRISP-DM from its origin to its current version;
(ii) its structure in terms of both process and methodology; (iii) an example of application and
(iv) a project report model. It is a summary of the following documents:
oCHAPMAN, P.; CLINTON, J.; KERBER, R.; KHABAZA, T.; REINARTZ, T.; SHEARER, C.;
WIRTH, [Link]-DM 1.0 Step-by-step data mining guide. 2001.
[Link]
[Link] SPSS Modeler CRISP-DM Guide. 2011.
[Link]
Other sources were:
oBARGER, D. Delivering Machine Learning Solutions. 2018.
[Link]
oFAYYAD, U., PIATETSKY-SHAPIRO, G., SMYTH, P. (1996). Knowledge Discovery and Data
Mining: Towards a Unifying Framework. KDD-96 Proceedings, Thirteenth National Conference
on Artificial Intelligence (AAAI-96), Association for the Advancement of Artificial Intelligence.
[Link]
oMicrosoft [Link] data science process.
[Link]
oPIATETSKY, [Link]-DM, still the top methodology for analytics, data mining, or data
science projects. KDnuggets, 2014.
[Link]
[Link]
oTAYLOR, [Link] Problems in Using CRISP-DM and How To Fix Them. KDNuggets, 2017.
[Link]

A simplified example, also presented in the IBM SPSS Modeler CRISP-DM Guide,
it accompanies the generic tasks of each phase. It relates to a specific application in retail
online. The actual product of a Data Science project is considerably more in-depth than
what is presented in the example.

Professors Edilson Ferneda & Hércules Antonio do Prado

Master's in Governance, Technology and Innovation
Catholic University of Brasília

CRISP-DM Guide 2
The discovery of knowledge

Until the late 1980s, the search for useful patterns (or models) in data was called Mining
Data, knowledge extraction, information discovery, information collection, archaeology
data and pattern processing. The term Data Mining was primarily used by
statisticians, data analysts, and management information system communities. Later,
In the mid-1990s, the concept of Data Mining established itself within the community of
Artificial Intelligence as the use of Machine Learning techniques to
the search for patterns in data sets within the context of Knowledge Discovery in Databases
Data.
The expression Knowledge Discovery in Databases (KDD), or Knowledge Discovery in Databases
Data Mining (DCBD) was coined by Gregory Piatetsky-Shapiro when, in 1989, he organized the first
Workshop on this topic, KDD-1989, during the main global event on Intelligence
Artificial, the International Joint Conferences on Artificial Intelligence (IJCAI). The goal was to emphasize
that "knowledge" is the final product of a data-driven discovery process. In 1996, in
another edition of this Workshop, Usama Fayyad, the same Gregory Piatetsky-Shapiro and Padhraic Smyth
proposes a methodology to guide the DCBD process, represented by Figure 1.

Figure 1: An overview of the stages that comprise the KDD Process, proposed by Fayyad, Piatetsky-Shapiro Smyth.

Source: Fayyad, U., Piatetsky-Shapiro, G., Smyth, P. (1996). Knowledge Discovery and Data Mining: Towards a Unifying
Framework. KDD-96 Proceedings, Thirteenth National Conference on Artificial Intelligence (AAAI-96), Association for
the Advancement of Artificial Intelligence.

The authors also point out that, before the start of the process, it is necessary to have a clear objective.
achieved, understand the scope of application and relevant prior knowledge and understand
this process from the customer's point of view.

The process involves: (i) selection of a dataset, (ii) processing of inconsistent data
or absent (data cleaning), (iii) identification of unnecessary data and inclusion of new ones
attributes (dimensionality reduction or transformation), (iv) application of one or more methods of
specific data mining to achieve the objectives of the process (data mining by
means of classification algorithms, regression, clustering, etc) and(v) Interpretation of patterns
obtained and, eventually, return to any of the previous stages for additional iteration. A
Once the discovered knowledge is consolidated, it must be incorporated into another system for other purposes.
actions, or documented/reported to stakeholders.
Meanwhile, the experience of applying this process in corporate environments has proven
insufficient, as it mainly reflects the activities to be developed by specialists in the areas of

CRISP-DM Guide 3
Computer Science and Data Analysis, not considering the ecosystem in which this process takes place.
it happens, the various actors involved and the material resources available for its execution.
In the quest for a more comprehensive methodology, in 1999, three companies, DaimlerChrysler,
interested in the application of data mining analyses in its business, NCR, a company
database solutions provider, and Statistical Product and Service Solutions, which
was marketing the SPSS software (Statistical Package for the Social Sciences), formed a consortium and
proposes the Cross Industry Standard Process for Data Mining (CRISP) methodology (or process)
Later, SPSS was acquired by IBM, which is now the company that disseminates the CRISP-
DM, incorporating it into your SPSS Modeler product.

Other similar methodologies have emerged, such as SEMMA (Sample, Explore, Modify, Model and
Assess) that focuses on modeling tasks and data mining projects, but without considering the
business aspects.
Figure 2 shows the poll presented in a text by Gregory Piatetsky-Shapiro.
published on the site Kdnuggets, about the most widespread DCBD methods in the corporate environment.
Since CRISP-DM has been widely used in data science projects not only in this environment,
but also in the scientific and academic fields, its adoption is justified for its dissemination among those interested in
Data Science.

Figure 2: Methodologies used in organizations for knowledge discovery projects in databases [200
votes.

2014 CRISP-DM 43.0%

42.0%
2007
27.5%
My own 19.0%

8.5%
SEMMA 13.0%

8.0%
Another, without specific domain 4.0%

7.5%
KDD Process 7,3%

3.5%
From my organization 5.3%

2.0%
Methodology for a specific domain 4.7%

0.0%
None 4,7%

Source: PIATETSKY, G. CRISP-DM, still the top methodology for analytics, data mining, or data science projects.
KDnuggets, 2014
[Link]

It is important to remember, however, that CRISP-DM has not undergone adjustments since its creation, and that it
own is an evolution of the KDD Process, from the mid-1990s, as it does not relate to the
process of knowledge discovery in the organizational context in which the project would be inserted.
Since then, technological advancements such as Big Data, Cloud Computing, Mobile Computing and
The Internet of Things enabled the emergence of decision support systems that consume
dynamically generated data in real time, coming from heterogeneous sources and often of low quality
quality, making them difficult to understand and process.
Given that DCBD performs offline analysis of data collected from databases, several proposals have
arisen to heal its limitations. Among the alternatives to CRISP-DM is the Science methodology

CRISP-DM Guide 4
de Dados ágil (e iterativa)Team Data Science Process(TDSP), proposta pela Microsoft, cujo ciclo de
life is presented in Figure 3. It was designed to compose projects for development of
smart applications that involve some type of predictive analysis. For targeted projects
only for data exploration or analysis, some phases can be omitted.

Figure 3: Life cycle of the Team Data Science Process (TDSP).

Source: Microsoft Azure. Team data science process.

[Link]

It is noted that this methodology is an evolution of CRISP-DM, by incorporating some modern practices.
of software engineering and enable the incorporation of new technologies. Thus, for beginners
in Data Science, mastering CRISP-DM is relevant, keeping in mind that this is
a first step for those who intend to delve deeper into this topic.

CRISP-DM Guide 5
The CRISP-DM

CRISP-DM can be considered from two aspects:

− Como uma metodologia, que inclui descrições das fases típicas de um projeto de DCBD, das
tasks involved in each phase and an explanation of the relationships between these tasks.
− Or as a process model, presenting an overview of the DCBD life cycle.
These two aspects complement each other in a guide for data scientists to develop applications.
data mining with the assurance of not omitting any relevant task.
The CRISP-DM life cycle model (Figure 4) consists of six phases with arrows indicating the
most important and frequent dependencies between the phases.

Figure 4: Life cycle of the CRISP-DM methodology/process.

Source: CHAPMAN, P. et al. CRISP-DM 1.0 Step-by-step data mining guide. 2001.
[Link]

The CRISP-DM model is flexible and can be easily customized. For example, if your organization
if you intend to detect money laundering, you are likely to examine large amounts of data
without a specific modeling goal. Instead of modeling, your work will focus on the
exploration and visualization of data to discover suspicious patterns in financial data. The CRISP-
DM allows you to create a data mining model that meets your specific needs.
In such a situation, the modeling, evaluation, and implementation phases may be less relevant.
that the phases of understanding and preparing the data. However, it is still important to consider
some of the issues raised during these later phases for long-term planning and
the future goals of data mining.
It is a recognized fact that data for data mining is rarely available in the form
desired. They are usually found in more than one file, which can even be external to the organization (by

CRISP-DM Guide 6
example, demographic files from IBGE). As a result, there is no guarantee that there are secure ways of
gather them, nor that the domains of their attributes or their formats are standardized. Thus,
as shown in Figure 5, Tasks that precede the actual data mining,
They usually consume most of the total effort to develop an application.

Figure 5: Typical proportion of effort for each phase of CRISP-DM in %.

40

35

30

25

20

15

10

0
Understanding Understanding Preparation of Modelagem Assessment Implementation
of the business two dice data

Source: BARGER, D. Delivering Machine Learning Solutions. 2018.

[Link]

Before starting with the phases of CRISP-DM, it is worth observing how the model is structured in terms
of the process. Figure 6 shows the layers of CRISP-DM when considered as a model of
processes. These layers are organized to guide the Data Scientist through their
various phases, with control of the more specific levels from the more general tasks, going through the
specific tasks up to the instances of processes where interventions are carried out.

Figure 6: Structure of CRISP

Source: CHAPMAN, P. et al. CRISP-DM 1.0 Step-by-step data mining guide. 2001.
[Link]

CRISP-DM Guide 7
1. BUSINESS UNDERSTANDING

At this stage, what the organization expects should be explored.

earn with the data mining project. It is recommended
involve people directly affected by the
project or holders of knowledge.
The rigor with which this phase is carried out will influence the quality.
of the first results and the number of iterations of the cycle of
life necessary to achieve the project results.

CRISP-DM Guide 8
1.1 Determining business objectives
The first generic task is to try to obtain as much information as possible about the objectives of
businesses for data mining, that is, the definition of the problem. This may not be so easy.
as much as it seems, it will help reduce later risks by making the problems, objectives and
clearer resources. The following scheme relates specific tasks, activities, and results
correspondents.

Compilation of basic information about the current business situation

Activities Result
Describe the organizational structure (organizational chart and projects)
corporate, with names and responsibilities, identifying the
stakeholders, sponsors, and sectors affected by the project
Describe the problem in general terms, the ecosystem (processes,
resources and actors) in which it is inserted, as well as the motivations and pre- Organizational context
requirements for the project, its status in the organization, and previous experiences
in data mining
Describe the current solution, its advantages, disadvantages, and level of
acceptance in the organization

Definition of business objectives by decision makers

Activities Result
Describe the business problem that one wants to address with Mining of
Data
Specify the issues associated with the problem Business objectives
Determine any other business requirements
Specify expected business benefits

Definition of project success criteria

Activities Result
Define the success criteria of the project
Define the success criteria for data mining for each of the Success criteria of
business objectives project from the perspective of
business
Define the assessors of subjective success measures. If possible,
note your expectations

Example:
As more companies transition to online sales, an established electronic retailer of
Computers/electronics face increasing competition from new websites.
Given the reality that online stores are emerging as quickly (or quicker!) as the
customers are migrating to the Web, the company must find ways to remain profitable despite
increasing customer acquisition costs. A proposed solution is to nurture existing relationships with the
client to maximize the value of each of the company's current clients.

CRISP-DM Guide 9
Thus, a study was commissioned with the following objectives: (i) improve cross-selling by
best recommendations and(ii)increase customer loyalty with a more personalized service.
As an attempt, the study will be deemed a success if: (i) cross-sales increase by 10%, (ii) customers
we will spend more time and consult more pages on the site per visit and (iii) the study is completed on time and
below the budget.

CRISP-DM Guide 10
1.2 Assessing the situation

With a clearly defined project objective in hand, it is time to assess where we stand.
we find, which involves answering what kind of data we have available for analysis and if we have the
necessary professionals to develop the project? What are the biggest risk factors
Involved? Do we have a contingency plan for each identified risk factor?

Inventory of resources
Activities Result
Raise available hardware and software resources
Identify sources of data and knowledge Inventory of resources
Identify technical and business personnel resources

Requirements gathering, assumptions, and constraints

Activities Result
Determine the requirements (e.g., for data security and for
project results
Clarify the assumptions (e.g., about economic factors, quality of Requirements, assumptions and
data and presentation of the results restrictions

Check restrictions (e.g. availability of passwords, financial, legal in

regarding access to the data

Identification of risks and contingencies

Activities Result
Identify possible risks (e.g., delays in project execution, problems
budgetary, data quality and less impactful results Risks and contingencies
Develop a contingency plan for each possible risk

Survey of the terminology used

Activities Result
Create a list of relevant technical and business terms or jargon
for the project Glossary
Share the list with the project team

Cost-benefit analysis
Activities Result
Identify the costs of obtaining the data (internal or external), of
implementation of results and operations
Costs and benefits
Identify the expected benefits of achieving the project's objectives
better understanding of the problem and the organization's data collection

CRISP-DM Guide 11
Example:
This is the first attempt of the electronic retail of electronics in mining on the Web and the company decided
consult a data mining specialist to help with the introduction. One of the first tasks with the
what the consultant faces is the evaluation of the company's resources for data mining.
Team. It is clear that there is internal knowledge regarding server log and database management.
product and purchase data, but little experience in data warehousing and data cleaning for analysis.
Thus, a database specialist should also be consulted. Given that the company expects that the
results of the study become part of a continuous web mining process, the management
It should also take into account whether any position created during the current effort will become permanent.

Data. Since this is an established company, there are sufficient web logs and purchase data in which to
to base. In fact, for this initial study, the company will restrict the analysis to customers who have 'registered' on the site. If
If successful, the program may be expanded.
Risks. In addition to the monetary expenses for the consultants and the time spent by employees on the study, not
there is a great immediate risk in this undertaking. However, time is always important, so this project
initial is planned for a single financial quarter.
Additionally, there is not a large cash flow surplus at the moment, so it is essential that the study takes place.
below the budget. If one of these goals is at risk, the business managers suggest that the scope of
the project is reduced.

CRISP-DM Guide 12
1.3 Determining the objectives of data mining
With the business objectives clear, it's time to translate them into data mining. For example, if
the business goal is to offer the most attractive price with acceptable profit, the mining targets of
dados poderiam ser:(i)identificar as fontes de precificação da concorrência,(ii)analisar os preços da
competition vis-à-vis our production cost and (iii) specify the acceptable profit margin taking into account
check the prices of the competition.

Definition of data mining objectives

Activities Result
Describe the type of data mining problem (e.g., clustering,
classification etc)
Objectives of mining
List the goals of data mining (e.g., valid predictions for three
data from the point of view
months) technical
Quantify the desired results in relation to the current situation (e.g.:
customer loss indicators

Definition of success criteria for data mining

Activities Result
Define and describe the methods for evaluating the model(s) (e.g.:
precision, performance, etc.
Define objective (numerical) performance criteria for evaluation of the Success criteria of the
success of data mining data mining

Specify subjective measures and determine those responsible for them.

["evaluations"]
Note: Ensure that the successful implementation of the model is part of the solution's success. Include the implementation in
planning.

Example:
With the help of his data mining consultant, the online retailer was able to convert the objects of
Company business in terms of data mining. The goals for the initial study to be completed in this
quarters are:
• Use historical information about previous purchases to generate a model that links 'related' items.
When users view an item description, provide links to other items in the related group.
(basket analysis).
• Use web logs to determine what different customers are trying to find and then design
again the site to highlight these items. Each different 'type' of customer will see a different homepage
for the site (profile determination).
• Use web logs to try to predict where the person will go next, given where they came from and if they have been in your
site (sequence analysis).

CRISP-DM Guide 13
1.4 Producing a project plan
Based on the results obtained so far, the project plan must be developed. A good
The project plan includes information on goals, resources, risks, and planning for all phases of the
data mining.

Development of the project plan from a technical perspective

Activities Result
Estimate the time necessary to complete each phase or activity
Estimate the efforts or resources required for the implementation of the Project plan of the point of
results for the business solution technical view
Define review and decision points in the plan.
Note: Consider that there are phases that require iteration, particularly the Model Building phase.

Assessment of tools and techniques

Activities Result
Evaluate the available tools
Tools and techniques
Evaluate the types of patterns produced through the chosen techniques adequate
(ex: neural network, decision tree, etc)

Example:
The overview plan of the study is as shown in the table below.

Phase Time Resources Risks

Understanding of 1 week All analysts Economic change
business
Understanding 3 weeks All analysts Data problems
two dice Technology problems
Preparation of the 5 weeks Mining consultant Data issues
data data, some time as Technology problems
database analyst
Modeling 2 weeks Mining consultant Technology problems
data, some time like Inability to reach a model
database analyst adequate
Assessment 1 week All analysts Economic change
Inability to implement the
results
Implementation 1 week Mining consultant of Economic change
data, some time like Inability to implement the
analista de banco de dados results

CRISP-DM Guide 14
2. DATA UNDERSTANDING

This phase corresponds to the stage of familiarization with the

data of the problem and identification of their quality,
obtaining the first perceptions and formulation of
prognoses about what the data may show.
The interaction of this phase with the first occurs due to necessity.
to confront the stated objectives with the data
available.

CRISP-DM Guide 15
2.1 Collecting initial data
At this point, we are ready to access the data. This data may already exist in the own
organization (transactional data, opinion research, web logs, etc.), external (such as data
demographic or economic) or additional (produced for the purpose of the project).

Initial data collection

Activities Results
Identify the potentially relevant databases for the project
For each base, identify the most promising attributes.
Identify attributes that seem irrelevant and that can be excluded
Check if there is enough data for generalizable conclusions or Definition process of the
accurate predictions initial data collection
Initial data collection
Check the need for dimensionality reduction of the data
If you are considering data from various sources, check the
feasibility or possible problems of merging them
Verificar como os valores faltantes poderão ser tratados
Note: Considering multiple data sources will require their integration, either at this stage or in the data preparation stage.
but later.

Example:
The online retailer in this example uses several important data sources, including:
Web logs. The raw access logs contain all the information about how customers navigate the
website. References to image files and other non-informative entries in the web logs will need to
to be removed as part of the data preparation process.
Purchase data. When a customer sends an order, all the information pertinent to that order is
salvas. As ordens no banco de dados de compra precisam ser mapeadas para as sessões correspondentes nos logs
from the Web.

Product database. Product attributes can be useful in determining 'related' products.

Product information needs to be mapped to the corresponding orders.
Customer database. This database contains additional information collected from customers.
registered. The records are not complete in any way, as several clients do not fill out the
Questionnaires. Customer information needs to be mapped to the corresponding purchases and sessions in
Web logs.
At this moment, the company has no plans to purchase external databases or to spend money.
conducting research, as their analysts are busy managing the data they currently have. At some
Meanwhile, they may wish to consider an extended implementation of mining results.
of data, in which the purchase of additional demographic data for unregistered customers can be very
Useful. It may be helpful to have demographic information to see how the electronic retailer's customer base differs.
of the average web buyer.

CRISP-DM Guide 16
2.2 Describing the data
There are several ways to describe the data, which includes its quantity and quality (how much data
are available and their characteristics). These characteristics include, primarily, the types of
each data item and the particular coding schemes.

Survey of information about the data

Activities Result
Check the formatting of the files
Identify the method used for data capture
Raise the size of the database(s) (number of rows and columns)
Description of the base(s)
Describe meaning, relationships, origin, use and type of attributes data
from each database (data dictionaries)
Verify the relevance of the data for achieving the project's objectives.
Calculate basic statistics for the key attributes

Example:
There are many records and attributes to be processed in a web mining application. Even though the
electronic retailer conducting this data mining project has limited the initial study to
Approximately 30,000 customers have registered on the site, there are still millions of records in the web logs.

Most types of values in these data sources are symbolic, whether they are dates and times, web pages.
accessed or answers to multiple choice questions in the registration questionnaire. Some of these variables
will be used to create new variables that are numeric, such as the number of web pages visited and the time
expense on the website. The few numerical variables present in the data sources include the number of each
requested product, the amount spent during a purchase and the specifications of weight and dimension of the product
product database.
There is little overlap in the coding schemes for the various data sources, as these data sources
contain very different attributes. The only overlapping variables are the 'keys', such as customer IDs and
product codes. These variables must have identical encoding schemes from data sources to sources
data, otherwise it would be impossible to merge the data sources. Some data preparation will be necessary.
additional to recode these main fields for merging.

CRISP-DM Guide 17
2.3 Exploring the data
In this phase, data is explored and analyzed by constructing tables, graphs, and other resources.
visualization with the available tools. These analyses can support the formulation of hypotheses and
the outlining of the data transformation tasks that occur during the preparation phase of the
data.

Data exploration implementation

Activities Result
Formulate hypotheses about the data
Identify promising attributes for analysis
Enumerate new characteristics about the data revealed by its
data exploration
Route and findings from
Report that changes in the initial hypotheses were caused by the from data exploration
data exploration
Identify particular subsets of the data for later use
Check if the result of the data exploration justifies the change of the
objectives of the project or data mining

Example:
Although CRISP-DM suggests conducting an initial exploration at this point, data exploration is difficult, if not impossible.
impossible, in raw web logs, as our electronic retailer discovered. Normally, the log data of the
Web should be processed first in the data preparation phase to produce data that can be
explored in a significant way. This departure from CRISP-DM emphasizes the fact that the process can and should
customized for your specific data mining needs. CRISP-DM is cyclical and the
data miners usually go back and forth between the phases.
Although web logs must be processed before exploration, there are other data sources available for the
Electronic retailers are more accessible for exploration. Using the purchase database for exploration reveals
interesting summaries about customers, such as how much they spend, how many items they purchase per transaction
and where they come from. The summarizations of the customer database will show the distribution of responses to
items in the registration questionnaire.

Exploration is also useful for seeking errors in the data. Although the largest data sources are generated
automaticamente, as informações no banco de dados de produtos foram inseridas manualmente.
Some quick summaries of listed product dimensions will help uncover typos such as
119 inch monitor

CRISP-DM Guide 18
2.4 Verifying data quality
Data is rarely perfect. Most data contains coding errors, missing values.
or other types of inconsistencies that hinder the achievement of data mining. It is recommended
a complete analysis of the quality of the available data before modeling. They must be
considered missing data, data errors, error metrics, consistent encodings and
incorrect metadata.

Data quality assessment

Activities Result
Verificar se há valores de atributo faltantes ou em branco e explicitar o que
this means
Check for spelling inconsistencies that may cause
problems in joins or transformations in the data
Check for deviations in the data values (outliers) and if this is noise or
Does it have any meaning?

Check the plausibility of the data values and record any eventual Report on the quality of
conflicts data

Check the possibility of deleting data that do not have an impact on

hypotheses
Check if the data are correctly separated by delimiters and if
there are the same number of fields in each line

Check if the quality of the data justifies the change of the objectives of
project or data mining

Example:
Data quality verification is usually performed during the processes of description and
exploration. Some of the problems encountered by the electronic retailer include:
• Missing data. The known missing data includes questionnaires unanswered by some of the
registered users. Without the additional information provided by the questionnaire, it may be necessary for these
clients may be left out of some of these subsequent models.
• Data errors. Most data sources are generated automatically, so this is not a big deal.
concern. Typographical errors in the product database may be found during the process of
exploration.
• Measurement errors. The greatest potential source of measurement error is the questionnaire. If any of the items are
poorly recommended or poorly formulated, it may not provide the information that the online retailer expects
obtain. Again, during the exploration process, it is important to pay special attention to items that have a
unusual distribution of responses.

CRISP-DM Guide 19
3. DATA PREPARATION

Here, tasks such as table selection and integration are carried out.
sampling, creation of new attributes, data cleaning
initial bruto, graph construction, elaboration of
dictionary of variables with their respective types and
partitioning of the file into training data and data
of test.
This phase depends on the databases identified in the phase
previous. On the other hand, it is interrelated with the techniques of
mining, defined in the first phase, and tools to be
used in the next phase.
At the end of this phase, the database should be available.
be used in the modeling phase.

CRISP-DM Guide 20
3.1 Selecting the data
Based on the initial data collection obtained in the previous phase, this is the time to select the data.
relevant to the objectives of data mining. This can occur in two ways: through the selection of
records (rows) or by selecting attributes (columns).

Rationale for the inclusion/exclusion of data

Activities Result
Check for the existence of especially relevant attributes for the
objectives of data mining
Identify if there are quality issues in a dataset or a
attribute that may compromise the validity of the results Inclusion criteria and
data exclusion applied
Check the possibility and ways to resolve these issues
Check if there are any restrictions regarding the use of specific attributes

Perform data cleaning

Example:
Many of the electronic retailer's decisions about which data to select have already been made in previous phases of
data mining process.
Select items. The initial study will be limited to approximately 30,000 customers who registered on the site.
therefore the filters need to be configured to exclude purchases and web logs from unregistered customers.
Other filters should be established to remove calls from image files and other non-entries.
informative in the Web logs.
Select attributes. The purchase database will contain confidential information about customers.
electronic retailer, therefore it is important to filter attributes such as name, address, phone number, and cards
of the customer's credit.

CRISP-DM Guide 21
3.2 Cleaning the data
This task involves a more detailed analysis of the problems in the selected data for analysis.
such as missing, inconsistent, or erroneous data. The various techniques for dealing with these
problems must be considered.

Data cleaning execution

Atividades Result
Relate the types of noises that occur in the data
Detail what approaches and techniques can be used to remove
these noises Strategies and actions for the
data cleaning
Perform data cleaning
Report if there were attributes with noise that could not be recovered

Example:
O varejista eletrônico usa o processo de limpeza de dados para abordar os problemas anotados no relatório de
data quality.
Missing data. It is likely that customers who did not complete the online questionnaire will have to be left out.
side of some models later on. It is possible to request again that these clients fill out the
questionnaire, but this will require time and money that the online retailer cannot afford to spend. The
what the online retailer can do is model the differences in purchases between customers who respond and those who do not
respond to the questionnaire. If these two groups of customers have similar shopping habits, the
Missing questionnaires are less concerning.
Data errors. The errors found during the exploration process can be corrected here. However,
in most cases, the appropriate data entry is imposed on the website before the customer submits a
page for the backend database.
Measurement errors. Items written unsatisfactorily in the questionnaire can greatly affect quality.
two dice. Just like with omitted questionnaires, this is a difficult problem, as there may not be time or
money available to collect responses for a new replacement question. For the problematic items, the
The best solution may be to return to the selection process and filter these items for additional analysis.

CRISP-DM Guide 22
3.3 Building new data
It is common to need to build new data by creating new attributes (columns) from
of already existing attributes or by the introduction of new records (rows).

Construction of new or derived data

Activities Result
Check if the data matches what is expected by the algorithm
chosen data mining and, if necessary, carry out the transformations
necessary
If necessary, perform the normalization of attributes.
Account on the derivation of
Define and apply approaches for handling missing data. new attributes
Report on the generation of
Verify the possibility of creating relevant attributes from the
new records
existing attributes
Check the need to apply oversampling or undersampling on
case of unbalanced bases
Generate new attributes and records

Example:
The processing of web logs can create several new attributes. For the events recorded in the logs, the
The online retailer will want to create date and time records, identify visitors and sessions, and note the page.
accessed and the type of activity that the event represents. Some of these variables will be used to create more
attributes, such as the time between events in a session.
Other attributes can be created as a result of a merging or other data restructuring.
For example, when the line-by-line event web logs are 'gathered' so that each line is a session,
New attributes will be created that record the total number of actions, the total time spent, and the total number of purchases made.
during the session. When the web logs are merged with the customer database so that each line
be a customer, new attributes will be created recording the number of sessions, the total number of actions, the time
spending is the total value of purchases made by each customer.

After building new data, the electronic retailer undergoes an exploration process to ensure that
that the data creation was executed correctly.

CRISP-DM Guide 23
3.4 Integrating the data
It is common to have the necessary data for data mining distributed across multiple sources. The
Basic methods for data integration are: importing columns or importing rows.
from different databases.

Implementation of data integration

Activities Result
Integrate (importing rows or columns) the datasets
considered useful for data mining
Account about the combination of
Generate a unique database for data mining data

Verify the need for the construction of new data

Example:
With various data sources, there are several different ways in which the online retailer can integrate the
data:
• Including customer and product attributes in event data. To model web log events using
attributes of other databases, each customer ID, product number, and purchase order number
associates to each event must be correctly identified and the corresponding attributes must be
merged in the processed Web logs. Note that the merged file replicates the customer information and
product every time a customer or product is associated with an event.
• Including purchase information and web logs in customer data. To model customer value,
your purchase and session information must be selected from the appropriate databases, totaled and
mixed with the client's database. This involves the creation of new attributes, as discussed in
data construction process.
After integrating the databases, the online retailer goes through an exploration process to
certify that the data merging was executed correctly.

CRISP-DM Guide 24
3.5 Formatting the data
As a final step before building the model, one must check if the modeling tool
requires a specific format for the data, such as, for example, the CSV format (Comma
Separated Values)

Data formatting execution

Activities Result
Check the necessary formats for using the tool of
chosen data mining Report on the formatting of the
data
Transport the database to the appropriate format

CRISP-DM Guide 25
4. MODELING

At this stage, a model is constructed based on the data, as

the type of task to be performed (grouping, classification,
association, etc) by a learning algorithm of
machine.
To initiate this phase, some issues must be
considered:
Are all the data accessible through the tool?
(ii) Based on the initial understanding and exploration of the data,
it was possible to select relevant subsets of
data?
Effective data cleaning or data removal was performed.
inconsistent?
(iv) The multiple databases were appropriately
integrated?
The requirements of the modeling tool are clear.
chosen?
Is there any formatting aspect to consider beforehand
from modeling?

CRISP-DM Guide 26
4.1 Selecting the modeling technique(s)
Even if there is already an idea of the types of models to be created, at this moment some decisions
definitive decisions regarding this must be made, taking into account: the types of data available in the database
data, the goals of mining, and the specific modeling requirements. They are usually
Various modeling techniques were tested until finding the one that produces the best results.
It may be necessary to return to the previous phase to make adjustments to the already processed data to ensure
the quality of the model to be built.

Evaluation of modeling techniques

Activities Result
Check if the model to be created requires data partitioning into
training and testing data
Check if there is enough data to produce a reliable model Modeling techniques
considered
Check if the data has sufficient quality to produce a
Adaptation of the data to
reliable model
premises concerning the
Check if the data types are appropriate for the model to be modeling techniques
built considered

Align the data with the needs of the data mining technique to be
used

Example:
The modeling techniques used by the electronic retailer are driven by mining goals.
company data:
• Improved recommendations. In the simplest terms, this involves purchase orders for storage in
cluster to determine which products are purchased together most frequently. Customer data, and even the
visit records can be included to achieve better results. The storage techniques
Kohonen network clusters are suitable for this type of modeling. Subsequently, the clusters can be
modeled using a set of C5.0 rules to determine which recommendations are the most appropriate
at any point during a client's visit.
• Improved website navigation. For now, the online retailer will focus on identifying
pages that are frequently used, but require several clicks for users to find them.
This requires applying a sequencing algorithm to the web logs in order to generate the 'unique paths'.
that customers can take from the website and then specifically look for sessions with many pages
will be visited without (or before) a measure is taken. Subsequently, in a deeper analysis, the
cluster storage techniques can be used to identify different 'types' of visits and
visitors and the website content can be organized and presented according to type.

CRISP-DM Guide 27
4.2 Designing the test design
At this stage, it should be detailed how the created models will be tested. There are two questions to be addressed.
considered: (i) what are the acceptance criteria for the model and
(ii) which data will be used in the model testing. The acceptance of a classification model, for
example, it can be done by a desired accuracy limit. Eventually, it may be necessary to
experimentation with different testing schemes, aiming for higher model quality.

Test design conception

Activities Result
Specify the test scheme of the model to be created (e.g.: use of
committee machines, separation of the database into training and testing,
parameters for this division, etc)
Test design
Define what the measure of success is for models
Define strategies for testing a model for its effectiveness in
regarding the objectives of data mining

Exemplo:
The criteria by which the models are evaluated depend on the models under consideration and the goals of
data mining
• Improved recommendations. Until the improved recommendations are available in real-time for the
customers, there is no purely objective way to evaluate them. However, the online retailer can require
that the rules that generate the recommendations are simple enough to make sense from a
business perspective. Likewise, the rules must be complex enough to generate
different recommendations for different clients and sessions.
• Improved website navigation. Given the evidence of which pages customers access on the website, the retailer
The electronic system can objectively evaluate the updated design of the website in terms of ease of access to pages.
important. However, just like the recommendations, it is difficult to assess in advance how the
clients will adjust to the reorganized site. If time and finances allow, some usability testing should
to be prepared.

CRISP-DM Guide 28
4.3 Building the model(s)
At this stage it is considered that all previous questions regarding an iteration of the process,
are resolved and the Data Scientist has all the conditions to build the desired model.
To arrive at this model, it is common to experiment with various model alternatives for the
same task and compare your results in order to choose the one that best fits the objectives of
mining.
The activities of this task must be carried out for each model under construction.

Structuring and construction of the model(s)

Activities Resultado
Calibrate the parameters of the algorithm that implements the technique of
chosen modeling Calibration process of the
Generate the model(s) parameters
Generated model(s)
Describe the results of the model(s), including indicators of
Descrição do(s) modelo(s)
performance, problems during execution and findings from the
exploration of the model (e.g.: new insights, useful patterns, problems of Results achieved
execution and inconsistencies

Example:
Improved recommendations. Clusterings are produced for variable levels of
data integration, starting only with the purchase database and then including the information
about the related client and the session. For each level of integration, the clusterings are produced under
variable parameter settings for the two-step network algorithms and Kohonen. For each one
From these clusterings, some sets of C5.0 rules are generated with different parameter configurations.
Improved site navigation. The Sequence modeling node is used to generate customer paths.
the algorithm allows for the specification of a minimum support criterion, which is useful for focusing on
most common paths of the customer. Various configurations for the parameters are tested.

CRISP-DM Guide 29
4.4 Evaluating the model(s)
With the generated models in hand, this is the time to determine the best ones, taking the criteria into account.
of successful mining defined in the Business Understanding phase.

Execution of the evaluation of the model(s)

Activities Result
Evaluate the results of the model(s), considering the indicators of
quality
Conduct a review of the results based on the understanding of the problem
of the business (from the perspective of Data Scientists and the specialists of
business)
Evaluation of the model(s)
Verify if the model results are easily usable in Foundation for the(s)
practice chosen model(s)
Analyze the impact of the model results considering the criteria
of successful mining defined earlier
Rank the model(s), choose the most suitable model(s) for
solve the business problem and justify this choice

Example:
Improved recommendations. One of the Kohonen networks and a two-step clustering provide results.
reasonable and the online retailer has difficulty choosing between them. Over time, the company hopes to use
both, accepting the recommendations that the two techniques combine and studying the situations in detail
in which they differ. With a little effort and applied business knowledge, the online retailer
You can develop other rules to address the differences between the two techniques.
The online retailer also discovers that the results that include session information are
surprisingly good. There is evidence that suggests that the recommendations may be linked to the navigation of
site. A set of rules, defining where the customer is likely to go next, can be used in real time
real to affect the website content directly while the customer is browsing.
Improved website navigation. The Sequence model provides the online retailer with a high level of confidence.
that certain customer pathways can be predicted, producing results that suggest a number
manageable changes in the website design.

CRISP-DM Guide 30
5. EVALUATION

At this moment, it is important to evaluate and review the steps.

executed to obtain the model that will allow the
scope of the project objectives.
Unsatisfactory results will lead to a return to the initial phase.
of the process for its restructuring.

CRISP-DM Guide 31
5.1 Evaluating the results
After verifying that the created models are technically correct and effective according to the
Success criteria for data mining that you defined earlier, the findings must be evaluated
of each model according to the business success criteria established at the beginning of the project.

Conducting an evaluation of the results

Activities Result
Verificar se os resultados apresentados estão inteligíveis
Check if there are particularly interesting findings that deserve
to be emphasized
Rank the models and findings according to their applicability to the Evaluation results of
business objectives mining
Select the model(s) that will be effectively used Selected models

Analyze to what extent the results address the business objectives

Identify additional issues that have arisen from the results,
regarding the business goals

Example:
The overall results of the electronic retailer's first experience with data mining are quite easy to
will be broadcast from a business perspective: the study produced what is expected to be
better product recommendations and an improved website design. The improved website design is based on
customer navigation sequences, which show the site features that customers want, but that
Several steps are required to achieve them. The evidence that the product recommendations are better
it is more difficult to transfer, as the decision rules can become complicated. To produce the final report,
The analysts will try to identify some general trends in the rule sets that can be explained.
more easily.
Ranking the models. As several of the initial models seemed to make sense in the business,
The ranking in this group was based on statistical criteria, ease of interpretation, and diversity.
Thus, the model provided different recommendations for different situations.
New questions. The most important question that arose from the study is: How can the online retailer know
more about your customers? The information in the customer database plays an important role in
formation of clusters for recommendations. Although special rules are available to make
recommendations for clients whose information is missing, the recommendations are more general in nature
as those made to registered clients.

CRISP-DM Guide 32
5.2 Reviewing the processes
Part of the CRISP-DM application consists of learning from the experienced gained so that future
data mining projects be more effective. It is necessary to summarize the decisions of each of the
phases completed. Thus, the activities of this task must be carried out for each of the phases/tasks
carried out.

Evaluation of the processes used during the project

Activities Result
Verify how the phase/task contributed to the final results
Check if there are ways to improve this phase/task
Identify the failures or errors of this phase/task and how to avoid them in the future.

Check if there were impasses (e.g., specific models that proved to be

inadequate) and if there are ways to predict such impasses in order to avoid Assessment of the process
waste of efforts
Identify positive or negative surprises during the execution of the
phase/task
Check for the existence of alternative decisions or strategies that could
to be used in a given phase/task

Example:
As a result of the review of the initial data mining project process, the online retailer
developed a greater appreciation of the interrelationships between the steps in the process. Initially reluctant to
"retrace" in the CRISP-DM process, the online retailer now sees that the cyclical nature of the process
increases its power. The review of the process also led the electronic retailer to understand that:
• A return to the exploration process is always justified when something unusual appears in another phase of
CRISP-DM process.
• Data preparation, especially of web logs, requires patience, as this can take a long time.
time.
• It is essential to stay focused on the business issues at hand, as soon as the data is ready for
Analysis, it is very easy to start building models without considering a broader picture.
• As soon as the modeling phase ends, understanding of the business becomes even more important in the decision of
how to implement results and determine which other studies are justified.

CRISP-DM Guide 33
5.3 Determining the next steps
At this moment, the decision is made to proceed with the implementation of the model or to return to
the Business Understanding phase for another iteration of the data mining process.

Definition of the next steps of the project

Activities Result
Evaluate the results of the mining. Decision on the next phase
Decide the next actions Lista de possíveis ações

Example:
The online retailer is very confident in the accuracy and relevance of the project's results and is therefore
proceeding to the implementation phase.
At the same time, the project team is also ready to go back and scale up some of the models so that
include predictive techniques. At this point, they are awaiting the delivery of the final reports and for a
green light from the decision-makers.

CRISP-DM Guide 34
6. IMPLEMENTATION

This is the phase in which the acquired knowledge is organized,

presented and put into use. Corresponds to the application of
new insights to make improvements in the organization. This can
to mean the creation of new processes or integration
formal of the model created to some existing process.

CRISP-DM Guide 35
6.1 Planning the implementation
At this moment, the Implementation Plan for the results of the mining should be elaborated.
data.

Conception of an implementation plan

Activities Result
Summarize the results (models and findings) for integration purposes with
information and sharing systems with the team
For each selected model, create a step-by-step plan for its
implementation and integration into the organization's processes

For each finding, create a dissemination plan along with the decision-makers.
decisions concerning
Implementation plan
If necessary, identify alternatives for the implementation of the model(s) Contingency plan
the findings

Specify how the use of the implemented results will be

monitored, including the definition of criteria to identify it
obsolescence
Elaborate a contingency plan for possible problems in
implementation

Example:
A successful implementation of the data mining results from the electronic retailer requires that the
correct information reaches the right people.
Decision makers. Decision makers need to be informed of recommendations and changes.
proposals for the site and receive brief explanations of how these changes will help. Assuming that they
once the study results are accepted, the people who will implement the changes need to be notified.
Web developers. The people who maintain the website will have to incorporate the new recommendations and the
organization of the website content. Inform them about the changes that may occur due to future studies,
so that they can lay the foundations now. Preparing the team for the rapid construction of the site based on the
real-time sequence analysis may be useful later.
Database specialists. The people who maintain customer, purchase, and product databases.
they must be informed about how the database information is being used and which attributes may be
included in databases in future projects.
Above all, the project team needs to keep in contact with each of these groups to coordinate the
implementation of results and planning future projects.

CRISP-DM Guide 36
6.2 Planning the monitoring and maintenance
Data mining work is continuous, which can lead to the restart of the CRISP life cycle.
When the solution starts to not meet the expectations of its users properly,
it becomes necessary to adjust it or, in more critical cases, to produce new models.

Conception of a monitoring and maintenance plan

Activities Result
Check for each model or finding what factors or influences are needed.
be tracked
Specify how the validity and accuracy of each model can be
measures and monitored
Define under what conditions the model(s) will be considered Monitoring plan and
obsolete maintenance

Define the measures to be taken when the obsolescence of the(s)

model(s)
Check if the considered model(s) can be obsolete.
used for similar business problems.
Note: It is here that good documentation becomes critical to assess the business objective of each mining project.
dados.

Example:
The immediate task for monitoring is to determine whether the organization of the new site and the recommendations
Improvements really work. That is, users can follow more direct paths to the pages they are on.
looking? Have the cross-sell of recommended items increased? After a few weeks of
through monitoring, the online retailer will be able to determine the success of the study.

What can be manipulated automatically is the inclusion of new registered users. When customers
They register on the site, the current rule sets can be applied to your information to determine which
recommendations they should receive.
Deciding when to update the rule sets to determine recommendations is a more delicate task.
Updating the rule sets is not an automatic process, as the creation of the cluster requires input.
manual regarding the appropriateness of a certain cluster solution.
Given that future projects generate more complex models, it is almost certain that the necessity and the amount of
monitoring will increase. When possible, the volume of monitoring should be automatic with reports
regularly planned available for review. Alternatively, the creation of models that provide
predictions quickly could be a direction the company would like to follow. This requires a greater
sophistication of the team than the first data mining project.

CRISP-DM Guide 37
6.3 Producing the final report
The preparation of a final report not only ties up the loose ends of the previous documentation, but
It can also be used to communicate your results. It is important to present your results to
the various people with an interest in the results (e.g., technical administrators responsible for
implementation of the modeling results or sponsors who will make decisions based on them
achieved results.

Presentation of results
Activities Result
Develop a complete description of the initial business problem
Describe the process developed for data mining
Detalhar os custos e os benefícios do projeto
Final report
Describe any deviations from the original project plan.
Final presentation
Develop a summary of the results of mining (models and findings)
Develop a general description of the implementation plan
Develop recommendations for future data mining work

Example:
The biggest deviation from the original project plan is also an interesting opportunity for greater work on
data mining. The original plan aimed to discover how to make customers spend more time and
to visualize more pages on the site per visit.
As you can see, keeping a customer happy is not simply a matter of keeping them online. The distributions of
frequency of time spent per session, divided into whether the session resulted in a purchase or not, they discovered
that the session times for most sessions resulting in purchases fall between the session times
of two groups of sessions without purchases.

Now that this is known, the problem is to find out if those customers who spend a lot of time on the site without buying
they are just looking without commitment or simply cannot find what they are
looking. The next step is to figure out how to deliver what they are looking for to encourage the
purchases.

CRISP-DM Guide 38
6.4 Reviewing the project
This is the final step of the CRISP-DM methodology and offers the opportunity to formulate your impressions.
finalize and group the lessons learned during the data mining process.

Production of a report on the lessons learned from the project

Activities Result
Detail your general impressions about the project
Detail what was learned from the project, both in relation to mining.
of data regarding the available data Experience report
Describe the parts of the project that deserve emphasis from the points of view
positive and negative

Example:
Interviews with project members. The online retailer discovers that the project members more
closely associated with the study, from start to finish, are mostly excited about the results and
they await future projects. The database group seems cautiously optimistic; although they appreciate the
usefulness of the study, they point to the additional load on the database resources. A consultant was
available during the study, but over time, another dedicated employee will be necessary.
database maintenance will be necessary as the project scope expands.
Interviews with clients. Customer feedback has been largely positive so far. One issue that has not
the impact of the website design change on established customers was well regarded. After some
Over the years, registered clients have developed certain expectations about how the site is organized. The feedback
the number of registered users is not as positive as that of unregistered customers and some really do not
they like the changes. The online retailer must pay attention to this problem and carefully consider whether
A change will bring enough new clients to risk losing the existing ones.

CRISP-DM Guide 39
RESULT IN ACTION!

After the first iteration of the project is completed, the model comes into use and must be monitored regarding its
effectiveness in relation to the initial problem, following the defined monitoring and maintenance plan
in the Implementation phase. The main task is to apply the knowledge obtained to generate results
of business and assess the quality of the model(s) over time.
Monitoring can be seen as a 'hidden' phase of CRISP-DM, as shown in Figure 7.

Figure 7: Hidden phases and transitions of CRISP-DM

When there is a loss of model quality or the current model can be improved from new ones.
data, for example, triggers a new cycle of the CRISP-DM process.

CRISP-DM Guide 40
Possible issues when using CRISP-DM

CRISP-DM is a great framework for data mining projects and its use helps to focus them on delivery.
real value to businesses. Being quite generic, many projects that use CRISP-DM adopt
some simplifications. Some simplifications make sense, but they can cause problems,
as shown in Figure 8.

Figure 8: Four possible problems when using a simplified version of CRISP-DM.

Source: TAYLOR, J. Four Problems in Using CRISP-DM and How To Fix Them. KDNuggets, 2017.
[Link]

Among the problems that may occur when using a simplified version of CRISP-DM are:
1. Lack of clarity. Instead of delving into details and gaining clarity about the problem of
businesses and how Data Science can help, the project team is satisfied with the goals of
businesses and some success metrics. The team now "understands" the business objective, wants
minimize the 'overload' and jump to the 'interesting' part of the project, analyzing the data. With
Often, this results in models that do not meet a real business need.
2. Irrational rework. Some Data Science teams simply evaluate the results.
of their projects in technical terms - if the model is predictive, it must be good. Most perceive
that this is not necessarily true and tries to verify your analytical results in relation to
business objective. This is difficult without a true clarity about the business problem. If the
the analysis they developed does not seem to meet the business objectives, the team has few
options. Most try to find new data or new modeling techniques, instead of
work with your business partners to reassess the business problem.
3. Blind transfer to IT. Some analytical teams do not consider the implementation and
operationalization of their models. Most perform better than this, recognizing that the
models they build will have to be applied to real data in databases
operational or incorporated into information systems. Even these teams usually do not

CRISP-DM Guide 41
 get involved with IT. Before that, they do not have clarity about the implementation needs of
results of the knowledge discovery process and do not consider implementation as a
data science work. The final result is a model handed over to IT without the necessary
care. Whether the model is easy or difficult (or even impossible) to implement or if it is
really usable after its implementation, this is a problem that does not concern them. This
increases the time and cost of implementing a model and contributes to the enormous
percentage of models that do not impact the business, that is, the process failed
as a whole.
4. Failure in iteration. Data Science professionals know that models age and that
they need to be updated in order to remain relevant. They know that the context of
business can change and decrease the value of a model. They know that data patterns
used for building the model may change. But they think of it as a problem
for another moment, that is, they do not have enough clarity about the business problem in order to
determine how to monitor the model's performance and do not worry about making the review
the least costly model in a possible new iteration of the process. After all, it is much more
It's interesting to face another new problem. This can lead to negligence regarding monitoring.
and the maintenance of the model(s), harming its long-term value.
Each of these problems increases the likelihood of the team creating an elegant analytical solution,
but that do not add value to the business. Organizations that truly want to explore Science
Data cannot allow the occurrence of these problems.

CRISP-DM Guide 42
DATA SCIENCE PROJECT

TÍTULO...........:xxxx

EQUIPE...........:xxxx (Nome, contato)

...
xxxx (Name, contact)

SUMMARY
1. UNDERSTANDING OF THE BUSINESS/PROBLEM
1.1 Determining business objectives
1.2 Evaluating the situation
1.3 Determining the objectives of data mining
1.4 Producing a project plan
2. DATA UNDERSTANDING
2.1 Collecting initial data
2.2 Describing the data
2.3 Exploring the data
2.4 Verifying the quality of the data
3. DATA PREPARATION
3.1 Selecting the data
3.2 Cleaning the data
3.3 Building new data
3.4 Integrating the data
3.5 Formatting the data
4. MODELING
4.1 Selecting the modeling technique
4.2 Designing the test design
4.3 Building the model(s)...........................................................................................................................................xx
4.4 Evaluating the model(s)...............................................................................................................................................xx
5. EVALUATION
5.1 Avaliando os resultados.................................................................................................................................................xx
5.2 Reviewing the processes
5.3 Determining the next steps
6. IMPLEMENTATION
6.1 Planning the implementation
6.2 Planning the monitoring and maintenance
6.3 Producing the final report
6.4 Reviewing the project

CRISP-DM Guide 43
1. BUSINESS UNDERSTANDING
1.1 Determining business objectives
Basic information about the current business situation
(i) Describe the organizational structure (organizational chart and corporate projects, with names and responsibilities, identifying the
stakeholders, sponsors, and sectors affected by the project;
xxx
(ii) Describe the problem in general terms, the ecosystem (processes, resources, and actors) in which it is embedded, motivations, and pre-
requirements for the project, its status in the organization, and previous experiences in data mining;
xxx
(iii) Describe the current solution, its advantages, disadvantages, and level of acceptance in the organization.

xxx
Business objectives from the decision-makers' perspective
Describe the business problem that you want to address with Data Mining;
xxx
Specify the issues associated with the problem;
xxx
(iii)Determinar quaisquer outros requisitos de negócio;(iv)Especificar benefícios esperados em termos de negócio.
xxx
Project success criteria
Define the success criteria of the project;
xxx
Define the success criteria of data mining for each business objective;
xxx
(iii) Define the evaluators of subjective success measures. If possible, note their expectations.
xxx

1.2 Assessing the situation

Resource inventory
Raise available hardware and software resources;
xxx
(ii) Identify data and knowledge sources; (iii) Identify resources of technical and business personnel.
xxx
Requirements, assumptions, and constraints
(i) Determine the requirements (e.g., regarding data security and project results);
xxx
(ii) Clarify the assumptions (e.g., about economic factors, data quality, and presentation of results);
xxx
(iii) Verify restrictions (e.g., availability of passwords, financial, legal regarding access to the data).
xxx
Risks and contingencies
(i) Identify possible risks (e.g., delays in project execution, budget issues, data quality, and poor results)
impactful)
xxx
(ii) Develop a contingency plan for each possible risk.

CRISP-DM Guide 44
 xxx
Survey of the terminology used
(i) Create a list of relevant technical and business terms or jargon for the project;
xxx
Share the list with the project team.
xxx
Cost-benefit analysis
Identify the costs of obtaining the data (internal or external), implementing the results, and operational costs;
xxx
(ii) Identify the expected benefits from achieving the project objectives (e.g., better understanding of the problem and the data collection)
organização).
xxx

1.3 Determining the objectives of data mining

Definition of data mining objectives
Note: Ensure that the successful implementation of the model is part of the solution's success. Include the implementation in
planning.

(i) Describe the type of data mining problem (e.g., clustering, classification, etc);
xxx
(ii) List the goals of data mining (e.g., valid predictions for three months);
xxx
(iii) Quantify the desired results in relation to the current situation (e.g., customer loss indicators).
xxx
Success criteria for data mining
(i) Define and describe the methods for evaluating the model(s) (e.g., accuracy, performance, etc.);
xxx
Define objective (numerical) performance criteria for assessing the success of data mining.
xxx
(iii) Specify subjective measures and determine the individuals responsible for the evaluations.

xxx

1.4 Producing a project plan

Project plan from a technical point of view
Note: Consider that there are phases that require iteration, particularly the Model Building phase.

Estimate the time required to carry out each phase or activity;

xxx
(ii) Estimate the efforts or resources needed for the implementation of the results for the business solution);
xxx
(iii) Define review and decision points in the plan.
xxx
Evaluation of tools and techniques
(i) Evaluate the available tools;
xxx
(ii) Evaluate the types of patterns produced through the chosen techniques (e.g., neural network, decision tree, etc.).
xxx

CRISP-DM Guide 45
 Synthesis of business understanding
xxx

2. UNDERSTANDING OF DATA
2.1 Collecting initial data
Initial data collection
Note: Considering multiple data sources will require their integration, either at this stage or in the data preparation phase.
front.

(i) Identify the potentially relevant databases for the project;

xxx
(ii) For each base, identify the most promising attributes.
xxx
(iii) Identify attributes that seem irrelevant and that can be excluded;
xxx
Check if there is enough data for generalizable conclusions or accurate predictions;
xxx
Check the need for dimensionality reduction of the data;
xxx
If you are considering data from various sources, check the feasibility or potential issues of merging them.
xxx
(vii) Verify how the missing values can be handled.
xxx

2.2 Describing the data

Information about the data
(i)Check the formatting of the files;
xxx
(ii) Identify the method used for data capture);
xxx
Raise the size of the database(s) (number of rows and columns);
xxx
(iv) Describe the meaning, relationships, origin, use, and type of the attributes of each database (data dictionaries);
xxx
Verify the relevance of the data for achieving the project's objectives;
xxx
(vi) Calculate basic statistics for the key attributes.
xxx

2.3 Exploring the data

Data exploration
(i) Formulate hypotheses about the data;

xxx
(ii) Identify promising attributes for analysis);
xxx

CRISP-DM Guide 46
 (iii) Enumerate new characteristics about the data revealed by your data exploration;
xxx
Report that changes in the initial hypotheses were caused by data exploration;
xxx
(v) Identify specific subsets of data for later use;
xxx
Verify if the result of the data exploration justifies the change of the project's objectives or the data mining.
xxx
Data quality
(i) Check for missing or blank attribute values and explain what that means;
xxx
(ii) Verify if there are spelling inconsistencies that may cause issues in data joins or transformations;
xxx
(iii) Check for deviations in data values (outliers) and whether this is noise or has some significance;
xxx
Check the plausibility of the data values and record any conflicts.
xxx
Verify the possibility of excluding data that does not impact the hypotheses;
xxx
(vi) Check if the data is correctly separated by delimiters and if there is the same number of fields in each line;
xxx
Verify if the quality of the data justifies changing the project's objectives or the data mining.
xxx

Synthesis of data understanding

xxx

3. DATA PREPARATION
3.1 Selecting the data
Rationale for the inclusion/exclusion of data
Verify the existence of especially relevant attributes for the objectives of data mining;
xxx
(ii) Identify if there are quality issues in a dataset or an attribute that may compromise the validity of the results);
xxx
(iii) Verify the possibility and ways to resolve these issues;
xxx
(iv) Check if there are any restrictions regarding the use of specific attributes; (v) Perform data cleansing.
xxx

3.2 Cleaning the data

Data cleaning
i) Relate the types of noise that occur in the data;
xxx

CRISP-DM Guide 47
 (ii) Detail which approaches and techniques can be used to remove these noises;
xxx
(iii) Perform data cleaning;
xxx
Report if there were attributes with noise that could not be recovered.
xxx

3.3 Building new data

Construction of new or derived data
Verify if the data is in accordance with what is expected by the chosen data mining algorithm and, if necessary, make the adjustments.
necessary transformations;
xxx
(ii) If necessary, perform attribute normalization);
xxx
(iii) Define and apply approaches for handling missing data;
xxx
Verify the possibility of creating relevant attributes from existing attributes;
xxx
Check the need to apply oversampling or undersampling in the case of unbalanced datasets;
xxx
Generate new attributes and records.
xxx

3.4 Integrating the data

Data integration
Integrate (import rows or columns) the datasets deemed useful for data mining;
xxx
(ii) Generate a unique database for data mining);
xxx
Verify the need for the construction of new data.
xxx

3.5 Formatting the data

Data formatting
Check the necessary formats for using the chosen data mining tool;
xxx
Transport the database to the appropriate format
xxx

Data preparation synthesis

xxx

4. MODELING
4.1 Selecting the modeling technique(s)
Evaluation of modeling techniques

CRISP-DM Guide 48
 Check if the model to be created requires partitioning the data into training and testing data;
xxx
(ii) Check if there is enough data to produce a reliable model);
xxx
(iii) Verify if the data has sufficient quality to produce a reliable model;
xxx
(iv) Verify if the data types are appropriate for the model to be built;
xxx
(v)Adequar os dados às necessidades da técnica de mineração de dados a ser utilizada.
xxx

4.2 Designing the test design

Test design
(i) Specify the test scheme for the model to be created (e.g., use of committee machines, separation of the database into
training and testing, parameters for this division, etc);
xxx
(ii) Define what the measure of success is for models);

xxx
(iii) Define strategies for testing a model regarding its effectiveness related to the objectives of data mining.
xxx

4.3 Building the model(s)

Structuring and construction of the model(s)
Calibration of the parameters of the algorithm that implements the chosen modeling technique;

xxx
(ii) Generate the model(s);
xxx
(iii) Describe the results of the model(s), including performance indicators, issues during execution, and findings from the
exploration of the model (e.g., new insights, useful patterns, execution problems, and inconsistencies).

xxx

4.4 Evaluating the model(s)

Evaluation of the model(s)
Evaluate the results of the model(s), considering the quality indicators;
xxx
(ii) Conduct a review of the results based on the understanding of the business problem (from the perspective of data scientists and the
business specialists);
xxx
(iii) Verify if the results of the models are easily applicable in practice;
xxx
(iv) Analyze the impact of the model results in light of the success criteria for mining defined earlier;
xxx
Rank the models, choose the most appropriate model(s) to solve the business problem, and justify this choice.
xxx

Synthesis of modeling
xxx

CRISP-DM Guide 49
5. EVALUATION
5.1 Evaluating the results
Evaluation of the results
Verify if the presented results are intelligible;
xxx
(ii) Verify if there are particularly interesting findings that deserve to be emphasized;
xxx
Rank the models and findings according to their applicability to the business objectives;
xxx
(iv) Select the model(s) that will be effectively used;
xxx
(v) Analyze to what extent the results respond to the business objectives;
xxx
Identify additional issues that have emerged from the results, in relation to the business goals.
xxx

5.2 Reviewing the processes

Evaluation of the processes used during the project
(i) Check how the phase/task contributed to the final results;
xxx
(ii) Verify if there are ways to improve this phase/task;
xxx
(iii) Identify the failures or errors of this phase/task and how to avoid them in the future;

xxx
(iv) Verify if there were deadlocks (e.g., specific models that proved inadequate) and if there are ways to predict such deadlocks.
to avoid wasting efforts;
xxx
Identify positive or negative surprises during the execution of the phase/task;
xxx
Verify the existence of alternative decisions or strategies that could be used in a given phase/task.
xxx

5.3 Determining the next steps

Next steps of the project
Evaluate the results of mining;
xxx
Decide the next actions.
xxx
Summary of the evaluation
xxx

6. IMPLEMENTATION
6.1 Planning the implementation

CRISP-DM Guide 50
 Implementation plan
(i) Summarize the results (models and findings) for integration with information systems and sharing with the team;
xxx
(ii) For each selected model, create a step-by-step plan for its implementation and integration into the organization's processes;
xxx
(iii) For each finding, create a dissemination plan with the relevant decision-makers;
xxx
(iv) If necessary, identify implementation alternatives for the model(s) and findings;
xxx
Specify how the use of the implemented results will be monitored, including the definition of criteria to identify their
obsolescence
xxx
Develop a contingency plan for possible problems during implementation.
xxx

6.2 Planning monitoring and maintenance

Monitoring and maintenance plan
Note: This is where good documentation becomes critical to assess the business objective of each data mining project.

Verify for each model or finding which factors or influences need to be tracked;
xxx
(ii) Specify how the validity and accuracy of each model can be measured and monitored;
xxx
(iii) Define under which conditions the model(s) will be considered obsolete;
xxx
Define the measures to be taken when the model(s) become obsolete;
xxx
Verify if the considered obsolete model(s) can be used for similar business problems.
xxx

6.3 Producing the final report

Results
Elaborate a complete description of the initial business problem;
xxx
(ii) Describe the process developed for data mining;
xxx
(iii) Detail the costs and benefits of the project;
xxx
Describe any deviations from the original project plan;
xxx
Prepare a summary of the mining results (models and findings);
xxx
Develop a general description of the implementation plan;
xxx
(vii) Develop recommendations for future data mining work.
xxx

CRISP-DM Guide 51
6.4 Reviewing the project
Lessons learned from the project
Detail your general impressions about the project;
xxx
(ii) Detail what was learned from the project, both in relation to data mining and in relation to the available data;
xxx
Describe the parts of the project that deserve highlighting from both positive and negative perspectives.

xxx

Summary of the implementation

xxx

CRISP-DM Guide 52