Chapter seven
Knowledge Discovery Systems
Systems That Create Knowledge
1
What is Knowledge Discovery
•Is the process of using computers to extract knowledge from
data.
•is a process of search new knowledge about an
application domain.
•Sometimes KD is also called Knowledge creation
which may be defined as the development of new tacit
or explicit knowledge from data and information or
from the mixture of prior knowledge.
2
What is Knowledge Discovery
KD is concerns the entire knowledge extraction
process including,
•How data are stored and accessed
•how to use efficient and scalable algorithms to
analyze massive datasets
•how to interpret and visualize the results
•how to model and support the interaction
between human and machine.
3
how knowledge is discovered?
Through socialization
Socialization enables the discovery of tacit knowledge
through joint activities between
boss and learners
researchers at an academic conference
Trough DM
by finding interesting patterns in observations, typically
embodied in explicit data.
4
knowledge discovery systems
The development of knowledge discovery systems has
already undergone three distinct phases.
first-generation
The systems is provide only one data mining
technique, such as
decision tree algorithm or a clustering algorithm.
It is very weak support for the overall process structure.
They planned for expert users who have already:
an understanding of data mining techniques and
Underlying/original data.
second-generation
•The systems used in this generation is called suites
or (integrated software package) or a collection of integrated
application running on a single program.
•They provide multiple types of integrated data
analysis methods, as well as
•It support for
data cleaning,
preprocessing, and
visualization.
third-generation
This system is addressed specific business problems,
such as
system detection, and
system discovery.
it provide an interface designed
to hide the internal complexity of data mining
methods.
It also introduce knowledge discovery process models
to guide the user’s work.
knowledge discovery processes
There are several reasons to structure a KDP as a
standardized process model:
•for the user/owner of the data.
•For a significant project management effort that
needs to be grounded in a solid framework.
•For planning and scheduling.
knowledge discovery in databases
The KDD field is concerned with the development of
methods and techniques for making sense of data.
•The (KDP),also known as knowledge discovery in
databases.
•KDD is the process of using data mining methods
(algorithms) to extract (identify) what is concidered to
be knowledge.
Data Mining and KDD
•A related field evolving from databases is data
warehousing, which refers to the popular business
trend of collecting and cleaning transactional data to
make them available for:
online analysis and
decision support.
Data Mining and KDD
Data warehousing helps set the stage for KDD in two
important ways:
data cleaning and
data access.
.
Data cleaning
•to address the issues of mapping data,
• representing and handling missing data, and
handling noise and errors.
Data access
•Uniform and well-defined methods must be created
for accessing the data and providing access paths to
data.
•A popular approach for analysis of data warehouses
is called online analytical processing.
•(OLAP)tools are targeted toward simplifying and
supporting for data analysis.
Knowledge Discovery from – Data Mining
• Another name for Knowledge Discovery in
Databases is data mining (DM).
• Data mining systems have made a significant
contribution in scientific fields of studies.
• The recent explosion of e-commerce applications,
providing reams/quantity of hard data ready for
analysis, presents us with an excellent oppurtunity
to make profitable use of data mining.
Knowledge Discovery from – Data Mining
• Data mining and knowledge discovery in databases
have been attracting a significant amount of
research, industry, and media attention of delayed.
• DM is a component of the KDD process that is
mainly concerned with means by which patterns
and models are extracted and enumerated from the
data.
• The DM and KDD are often used interchangeably
• DM is a more common term in business world, and
KDD in academic world.
Data Mining
•Data mining is the analysis of (often large)
observational data sets to find unsuspected
relationships an to summarize the data in novel ways
that are both understandable and useful to the data
owner.
•Data mining is a step in the KDD process consisting of
particular data mining algorithms that, under some
acceptable computational efficiency limitations,
produces a particular enumeration of patterns over
database.
Data Mining Techniques Applications
Marketing
• Predictive DM techniques, like artificial neural networks (ANN), have
been used for target marketing including market segmentation.
Direct marketing
• customers are likely to respond to new products based on their
previous consumer behavior.
Retail
• DM methods have similarly been used for sales forecasting.
Market container analysis
• uncover which products are likely to be purchased together.
Data Mining Techniques Applications
Banking
• Trading and financial forecasting are used to determine derivative
securities pricing, futures price forecasting, and stock performance.
Insurance
• DM techniques have been used for segmenting customer groups to
determine premium pricing and predict claim frequencies.
Telecommunications
• Predictive DM techniques have been used to attempt to reduce
churn, and to predict when customers will attrition to a competitor.
Operations management
• Neural network techniques have been used for planning and
scheduling, project management, and quality control.
Designing the Knowledge Discovery System – CRISP DM
(CRoss-Industry Standard Process for Data Mining)
The CRISP-DM KDP model consists of six steps, which
are
1. Business understanding.
• This step focuses on the understanding of objectives
and requirements from a business perspective.
• It is further broken into several sub steps,
determination of business objectives,
assessment of the situation,
determination of DM goals, and
generation of a project plan.
2. Data understanding.
•This step starts with initial data collection and
familiarization with the data. Specific aims include
identification of data quality problems, initial insights into the
data, and detection of interesting data subsets.
•Data understanding is further broken down into
collection of initial data,
description of data,
exploration of data, and
verification of data quality.
3. Data preparation
•needed to construct the final dataset, which constitutes the
data that will be provide for DM tool
•It includes Table, record, and attribute selection; data
cleaning; construction of new attributes; and transformation
of data. It is divided into
selection of data,
distillation of data,
construction of data,
integration of data, and
formatting of data
4. Modeling.
•At this point, various modeling techniques are
selected and applied.
•This step is subdivided into
selection of modeling technique(s),
generation of test design,
creation of models, and
assessment of generated models.
5. Evaluation.
•At the end of this phase, a decision about the use of
the DM results should be reached. The key substeps in
this step include
evaluation of the results,
process review, and
determination of the next step.
6. Deployment.
•Depending on the requirements, this step can be as simple as
generating a report or as complex as implementing a
repeatable KDP. This step is further divided into
plan deployment,
plan monitoring and maintenance,
generation of final report, and
n d
e e
T h
26