Data Science - BCA VI Semester

Assignment Questions & Answers

Q1. What is Data Mining?

Definition: Data mining is the process of analyzing large datasets to discover patterns,
relationships, correlations, or meaningful insights that can help in making informed
decisions and predictions.
Purpose: The primary purpose of data mining is to extract valuable knowledge and
information from large volumes of data that might be hidden or not readily apparent. It
involves using advanced statistical and machine learning techniques to identify patterns
and trends.
Functions: Data mining algorithms and techniques are applied to the data to identify
associations, clusters, classifications, and anomalies. It helps in understanding
customer behavior, predicting trends, detecting fraud, and making data-driven business
decisions.
Usage: Data mining is widely used in areas such as marketing analysis, customer
segmentation, recommendation systems, fraud detection, healthcare research, and
financial forecasting.
Tasks of Data Mining: Classification (categorizing data into predefined classes),
Clustering (grouping similar data points together), Regression (predicting numerical
values), Association Rule Mining (discovering relationships between variables),
Anomaly Detection, Text Mining, Prediction and Forecasting, and Pattern Mining.

Q2. What is KDD (Knowledge Discovery in Databases)? Explain the

relationship between KDD and Data Mining.
KDD Definition: KDD (Knowledge Discovery in Databases) is a computer science field
specializing in extracting previously unknown and interesting information from raw data.
KDD is the whole process of trying to make sense of data by developing appropriate
methods or techniques.
Steps included in the KDD process:
 Data Cleaning: Removal of noisy and irrelevant data from collection.
 Data Integration: Heterogeneous data from multiple sources combined in a
common source (Data Warehouse).
 Data Selection: Process where data relevant to the analysis is decided and
retrieved from the data collection.
 Data Transformation: Transforming data into appropriate form required by
mining procedure.
 Data Mining: Techniques that are applied to extract patterns potentially useful.
 Pattern Evaluation: Identifying strictly increasing patterns representing
knowledge based on given measures.
  Knowledge Representation: Presenting the results in a way that is meaningful
and can be used to make decisions.
Relationship between KDD and Data Mining: Data Mining is a step within the larger
KDD process. KDD is the overall process of discovering useful knowledge from data,
while Data Mining refers specifically to the extraction of patterns from data. In other
words, Data Mining is the core analytical step of KDD, whereas KDD encompasses the
entire pipeline from data preparation to knowledge representation.

Q3. Differentiate between KDD and Data Mining.

Parameter KDD Data Mining
Definition Process of identifying valid, Process of extracting useful and
novel, potentially useful and valuable information or patterns
understandable patterns in from large data sets.
data.
Objective To find useful knowledge from To extract useful information
data. from data.
Techniques Data cleaning, data integration, Association rules, classification,
Used data selection, data clustering, regression, decision
transformation, data mining, trees, neural networks, and
pattern evaluation and dimensionality reduction.
knowledge representation.
Output Structured information such as Patterns, associations, or
rules and models that can be insights that can be used to
used to make decisions or improve decision-making.
predictions.
Focus Discovery of useful knowledge Discovery of patterns or
rather than simply finding relationships in data.
patterns.
Role of Domain expertise is important in Domain expertise helps in
Domain defining goals, choosing data, choosing appropriate data and
Expertise and interpreting results. interpreting results.

Q4. What is the difference between DBMS and Data Mining?

Scope: DBMS focuses on efficiently managing and storing data, ensuring data integrity
and security. Data mining focuses on analyzing data to discover meaningful patterns
and insights.
Purpose: DBMS is used for data storage, retrieval, and management. Data mining is
used for knowledge discovery and gaining insights from the data.
Functionality: DBMS provides functionalities for data storage, retrieval, and
manipulation. Data mining employs algorithms and statistical techniques to identify
patterns and relationships within the data.
Role: DBMS serves as the foundation for data storage and retrieval, enabling efficient
data handling. Data mining is a process that builds on top of the data stored in the
DBMS to extract valuable information.
Summary: DBMS is the infrastructure for storing and managing data, while data mining
is a process of analyzing and extracting knowledge from the data stored in the DBMS.

Q5. Explain briefly the Data Mining Techniques.

Data mining techniques are algorithms and methods used to extract information and
insights from data sets:
 Regression: Used to model the relationship between a dependent variable and
one or more independent variables. Used to make predictions or forecasts.
Types include linear, logistic, and non-linear regression.
 Classification: Used to predict the class or category of an item based on its
characteristics. Models include decision trees, k-nearest neighbours, and support
vector machines.
 Clustering: Used to group items in a dataset into clusters based on their
similarity or proximity. Algorithms include k-means, hierarchical, and density-
based clustering.
 Association Rule Mining: Used to identify and explore relationships between
items in a dataset. Algorithms include Apriori and FP-growth.
 Dimensionality Reduction: Used to reduce the number of features in a dataset
while retaining as much information as possible. Methods include PCA, ICA, and
SVD.
 Anomaly Detection: Identifies outliers or anomalies in data that deviate from
normal patterns. Used for fraud detection, network intrusions, and equipment
failures.
 Sequential Pattern Mining: Discovers patterns that occur sequentially or
temporally in data.
 Text Mining: Extracts useful information from unstructured text data using NLP
and machine learning.

Q6. What are the problems in Data Mining?

 Data Quality: The accuracy, completeness, and consistency of the data affect
the accuracy of results. Data may contain errors, omissions, duplications, or
inconsistencies leading to inaccurate results.
 Data Complexity: Vast amounts of data from sensors, social media, and IoT are
difficult to process, analyze, and understand. Data may be in different formats,
making it challenging to integrate.
 Data Privacy and Security: As more data is collected, the risk of data breaches
and cyber-attacks increases. Regulations such as GDPR, CCPA, and HIPAA
impose strict rules on data collection, use, and sharing.
  Scalability: Data mining algorithms must be scalable to handle large datasets
efficiently. As dataset size increases, the time and computational resources
required also increase.
 Interpretability: Data mining algorithms can produce complex models that are
difficult to interpret, as they use a combination of statistical and mathematical
techniques.

Q7. Discuss the issues and challenges in Data Mining.

 Data Quality Issues: Real-world data tends to be incomplete, noisy, and
inconsistent. Practitioners must apply data cleaning and preprocessing
techniques to improve quality.
 Data Complexity: Data from various sources may be in different formats,
making integration difficult. Advanced techniques such as clustering,
classification, and association rule mining are needed.
 Privacy and Security: Data may contain personal, sensitive, or confidential
information. Anonymization and encryption techniques must be applied.
 Scalability: Distributed computing frameworks such as Hadoop and Spark are
used to address scalability challenges.
 Interpretability: Complex models are difficult to interpret. Visualization
techniques are used to represent data and models visually.
 High Dimensionality: Large numbers of features can make analysis difficult and
computationally expensive, requiring dimensionality reduction.

Q8. Explain the applications of Data Mining.

 Businesses and Enterprises: Many businesses use data mining to extract
useful insights and information from their data, in order to make better decisions,
improve their operations, and gain a competitive advantage. For example, a retail
company might use data mining to identify customer trends and preferences or to
predict demand for its products.
 Government Agencies and Organizations: Government agencies use data
mining to analyze data related to their operations and the population they serve.
A health department might use data mining to identify patterns and trends in
public health data or to predict the spread of infectious diseases.
 Academic and Research Institutions: Academic and research institutions use
data mining to analyze data from their research projects and experiments, in
order to identify patterns, relationships, and trends.
 Individuals: Many individuals use data mining to analyze their own data, in order
to better understand and manage their personal information and activities.
 Other Applications: Marketing analysis, customer segmentation,
recommendation systems, fraud detection, healthcare research, financial
forecasting, and fraud detection.
Q9. What is a Data Warehouse?
A Data Warehouse (DW) is a relational database that is designed for query and analysis
rather than transaction processing. It includes historical data derived from transaction
data from single and multiple sources.
A Data Warehouse provides integrated, enterprise-wide, historical data and focuses on
providing support for decision-makers for data modeling and analysis.
Attributes of a Data Warehouse:
 It is a database designed for investigative tasks, using data from various
applications.
 It supports a relatively small number of clients with relatively long interactions.
 It includes current and historical data to provide a historical perspective of
information.
 Its usage is read-intensive.
 It contains a few large tables.
Benefits of Data Warehouse:
 Understand business trends and make better forecasting decisions.
 Designed to perform well with enormous amounts of data.
 More accessible for end-users to navigate, understand, and query.
 Efficient method to manage demand for lots of information from lots of users.
 Provides capabilities to analyze a large amount of historical data.

Q10. Explain the Multidimensional Data Model.

The dimensions are the perspectives or entities concerning which an organization
keeps records. For example, a shop may create a sales data warehouse to keep
records of the store's sales for the dimension time, item, and location.
A multidimensional data model is organized around a central theme, for example, sales.
This theme is represented by a fact table. Facts are numerical measures. The fact table
contains the names of the facts or measures of the related dimensional tables.
Fact Table: The central table in a multidimensional data model that stores numerical
measures (facts) and keys to each of the dimension tables.
Dimension Table: Each dimension has a related table called a dimensional table,
which describes the dimension further. For example, a dimensional table for an item
may contain the attributes item_name, brand, and type.
Data Cube: Data can also be represented as a 3D data cube. When data is grouped or
combined in multidimensional matrices, they are called Data Cubes. For example, sales
data can be viewed across three dimensions: time, item type, and location.
OLAP (Online Analytical Processing): OLAP is a computing method that allows users
to extract useful information and query data from different angles. It stores data in
multidimensional databases and supports complex calculations, time intelligence, and
business-focused calculations.
Q11. Write short notes on the following:
a) Data Cleaning
Real-world data tends to be incomplete, noisy, and inconsistent. Data cleaning routines
attempt to fill in missing values, smooth out noise while identifying outliers, and correct
inconsistencies in the data.
Data cleaning tasks include:
 Fill in missing values: The tuple is ignored, values are filled manually, or the
attribute mean fills the missing values.
 Identify outliers and smooth out noisy data: Using Binning, Regression,
Clustering, and Combined computer and human inspection methods.
 Correct inconsistent data: Inconsistencies can be recorded in various
transactions, during data entry, or arising from integrating information from
multiple databases.
b) Data Integration
Data integration is one of the steps of data pre-processing that involves combining data
residing in different sources and providing users with a unified view of these data.
 It merges the data from multiple data stores (data sources).
 It includes multiple databases, data cubes or flat files.
 Metadata, Correlation analysis, data conflict detection, and resolution of semantic
heterogeneity contribute towards smooth data integration.
 Tight Coupling: Data is pulled from different sources into a single physical
location through the process of ETL (Extraction, Transformation and Loading).
 Loose Coupling: A virtual mediated schema provides an interface that takes the
query from the user, transforms it, and sends it directly to the source databases.
c) Data Transformation
Data Transformation is defined as the process of transforming data into appropriate
form required by mining procedure. It is used to convert data from a source data format
into destination data.
 Smoothing: Works to remove noise from the data using binning, regression, and
clustering.
 Aggregation: Summary or aggregation operations are applied to the data. This
step is used in constructing a data cube for analysis of data at multiple
granularities.
 Generalization: Low-level or 'primitive' raw data are replaced by higher-level
concepts through the use of concept hierarchies. E.g., age may be mapped to
higher-level concepts like youth, middle-aged, and senior.
 Normalization: Attribute data are scaled to fall within a small specified range,
such as 0 to 1. Three methods: min-max normalization, z-score normalization,
and normalization by decimal scaling.
 Attribute Construction: New attributes are constructed and added from the
given set of attributes to help the mining process.
d) Data Reduction
Data reduction is a method of reducing the size of original data so that it may be
represented in a much smaller space. While reducing data, data reduction techniques
preserve data integrity.
Data Reduction Techniques:
 Dimensionality Reduction: Removes characteristics from the data set, resulting
in a reduction in the size of the original data. Methods: Wavelet Transform and
Principal Component Analysis.
 Numerosity Reduction: Decreases the size of the original data and expresses it
in a more compact format. Parametric (stores just data parameters) and Non-
Parametric (Histogram, Clustering, Sampling) approaches.
 Data Cube Aggregation: Condenses data into a more manageable format
through multidimensional aggregation.
 Data Compression: Process of altering, encoding, or transforming the structure
of data in order to save space. Includes lossless and lossy compression.
e) Discretization
Data discretization is a technique for dividing continuous nature qualities into data with
intervals. We use labels of tiny intervals to replace several of the characteristics'
constant values. This implies that mining results are presented in a clear and succinct
manner.
 Top-down Discretization (Splitting): If the process begins by first discovering
one or a few points (known as split points or cut points) to split the whole attribute
range, and then continuing this recursively on the resulting intervals.
 Bottom-up Discretization (Merging): Starts by considering all of the continuous
values as potential split-points, removes some by merging neighbourhood values
to form intervals, and then recursively applies this process to the resulting
intervals.

Q12. Define Pattern in Data Mining.

A pattern in data mining refers to a recurring structure, relationship, or regularity that
can be discovered within a dataset. Patterns are the output of the data mining process
and represent knowledge extracted from raw data.
Pattern Evaluation: Pattern Evaluation is defined as identifying strictly increasing
patterns representing knowledge based on given measures. It finds an interestingness
score of each pattern, and uses summarization and visualization to make data
understandable by the user.
Types of Patterns:
 Frequent Patterns: Item sets, subsequences, or substructures that appear
frequently in a data set.
 Association Patterns: Rules that describe the co-occurrence or occurrence of
items in a dataset.
 Classification Patterns: Patterns that categorize data into predefined classes.
  Sequential Patterns: Patterns that occur sequentially or temporally in data.

Q13. What is meant by Data Explosion?

Data Explosion refers to the rapid and exponential growth in the volume of data being
generated, collected, and stored in the digital world. With the advent of the internet, IoT
(Internet of Things), social media, sensors, and mobile devices, an enormous amount of
data is being created every second.
Causes of Data Explosion:
 Social media platforms generating millions of posts, images, and videos daily.
 IoT devices producing continuous streams of sensor data.
 Digital transactions and e-commerce activities.
 Scientific research generating massive datasets.
Data Explosion directly motivates the need for data mining, as traditional data analysis
methods are insufficient to handle such large volumes of data. Data mining techniques
and tools are needed to extract meaningful insights from this explosion of data. The
availability of large data sets and the growth of cloud computing and big data
technologies have made data mining even more powerful and widely used.

Q14. Write two advantages of Data Mining.

1. Improved Decision-Making: Data mining can provide valuable insights that can help
organizations make better decisions by identifying patterns and trends in large data
sets. It allows organizations to uncover insights that would be difficult or impossible to
discover manually, helping them improve their operations and gain a competitive
advantage.
2. Fraud Detection: Data mining can be used to identify fraudulent activities by
detecting unusual patterns and anomalies in data. It can also be used to identify new
product opportunities by analyzing customer purchase patterns and preferences, to
build predictive models that can forecast future events and trends, and to help
organizations better understand their customers and tailor their products and services to
meet their needs.

Q15. What are the sources of Data Mining?

Data mining works on data collected from various sources. The major sources of data
for data mining are:
 Databases: Relational databases, transactional databases, object-relational
databases, and data warehouses are primary sources.
 Data Warehouses: Repositories that store integrated, enterprise-wide, historical
data collected from multiple operational systems.
 World Wide Web (WWW): Web pages, web logs, and web usage data are rich
sources for web mining.
 Other Data Repositories: Flat files, spreadsheets, text documents, images,
videos, and sensor data.
  Social Media: Platforms like Twitter, Facebook, and LinkedIn generate large
volumes of data about user behavior and preferences.
 IoT Devices: Sensors and connected devices continuously generate data about
physical environments.
 Domain-Specific Applications: Finance, telecommunications,
DNA/bioinformatics, stock markets, and e-mail systems.

Q16. Explain the KDD Process with Steps.

The KDD (Knowledge Discovery in Databases) process is a systematic approach to
discovering useful knowledge from large amounts of data. The following steps are
included in the KDD process:
Step 1 - Data Cleaning: Data cleaning is defined as removal of noisy and irrelevant
data from the collection. This includes cleaning in case of missing values, cleaning
noisy data (where noise is a random or variance error), and cleaning with data
discrepancy detection and data transformation tools.
Step 2 - Data Integration: Data integration is defined as heterogeneous data from
multiple sources combined in a common source (Data Warehouse). It uses Data
Migration tools, Data Synchronization tools, and the ETL (Extract-Load-Transformation)
process.
Step 3 - Data Selection: Data selection is the process where data relevant to the
analysis is decided and retrieved from the data collection. For this we can use Neural
network, Decision Trees, Naive Bayes, Clustering, and Regression methods.
Step 4 - Data Transformation: Data Transformation is the process of transforming
data into appropriate form required by mining procedure. This is a two-step process:
Data Mapping (assigning elements from source base to destination to capture
transformations) and Code Generation (creation of the actual transformation program).
Step 5 - Data Mining: Data mining is defined as techniques that are applied to extract
patterns potentially useful. It transforms task-relevant data into patterns, and decides
purpose of model using classification or characterization.
Step 6 - Pattern Evaluation: Pattern Evaluation is defined as identifying strictly
increasing patterns representing knowledge based on given measures. It finds an
interestingness score of each pattern and uses summarization and visualization to
make data understandable by the user.
Step 7 - Knowledge Representation: This involves presenting the results in a way that
is meaningful and can be used to make decisions. The discovered knowledge is
presented to users in a comprehensible format such as reports, visualizations, or
dashboards to aid in decision making.

Fundamentals of Data Science | BCA VI Semester | Karnataka University