Experiment 1: Installation of WEKA Tool

Aim: A. Investigation the Application interfaces of the Weka

tool. Introduction:
Introduction
Weka (pronounced to rhyme with Mecca) is a workbench that
contains a collection of
visualization tools and algorithms for data analysis and
predictive modeling, together with
graphical user interfaces for easy access to these functions.
The original non-Java version of
Weka was a Tcl/Tk front-end to (mostly third-party) modeling
algorithms implemented in other
programming languages, plus data preprocessing utilities in C,
and Make file-based system forrunning machine learning
experiments. This original version was primarily designed as
a tool foranalyzing data from agricultural domains, but the
more recent fully Java-based version (Weka 3),for which
development started in 1997, is now used in many different
application areas, inparticular for educational purposes and
research. Advantages of Weka include:
Free availability under the GNU General Public License.
Portability, since it is fully implemented in the Java
programming language and thus runs
on almost any modern computing platform
A comprehensive collection of data preprocessing and
modeling techniques
Installation of Weka
You can download Weka from the official website
[Link]
Execute the following commands at the command prompt to
set the Weka environment variable for Java, as follows:
setenv WEKAHOME /usr/local/weka/weka-3-0-
2
setenv CLASSPATH
$WEKAHOME/[Link]:$CLASSPATH
Once the download is completed, run the exe file and choose
the default set-up.
Weka application interfaces
There are totally five application interfaces available for
Weka. When we open Weka, it will start the Weka GUI
Chooser screen from where we can open the Weka application
interface.
Weka data formats
Weka uses the Attribute Relation File Format for data
analysis, by default. But listed below are some formats that
Weka supports, from where data can be imported:
 CSV
 ARFF
 Database using ODBC

Attribute Relation File Format (ARFF)

This has two parts:

1) The header section defines the relation (data set) name,
attribute name and the type.
2) The data section lists the data instances.
An ARFF file requires the declaration of the relation, attribute
and data. Figure 3 is an example of an ARFF file.
· @relation: This is the first line in any ARFF file, written in
the header section, followed by the relation/data set name. The
relation name must be a string and if it contains spaces, then it
should be enclosed between quotes.
· @attribute: These are declared with their names and the
type or range in the header section. Weka supports the
following data types for attributes:
 Numeric
 <nominal-specification>
 String
 date
  @data – Defined in the Data section followed by the list
of all data segments
Weka Explorer
The Weka Explorer is illustrated in Figure 4 and contains a
total of six tabs.
The tabs are as follows.
1) Preprocess: This allows us to choose the data file.
2) Classify: This allows us to apply and experiment with
different algorithms on preprocessed data files.
3) Cluster: This allows us to apply different clustering tools,
which identify clusters within the data file.
4) Association: This allows us to apply association rules,
which identify the association within the data.
5) Select attributes: These allow us to see the changes on the
inclusion and exclusion of attributes from the experiment.
6) Visualize: This allows us to see the possible visualisation
produced on the data set in a 2D format, in scatter plot and bar
graph output.
The user cannot move between the different tabs until the
initial preprocessing of the data set has been completed.
Preprocessing

Data preprocessing is a must. There are three ways to inject

the data for preprocessing:
 Open File – enables the user to select the file from the
local machine
 Open URL – enables the user to select the data file from
different locations
Experiment 2:Implementing data preprocessing.

Data Preprocessing in Data Mining

Preprocessing in Data Mining:
Data preprocessing is a data mining technique which is used
to transform the raw data in a useful and efficient format.

1. Data Cleaning:
The data can have many irrelevant and missing parts. To
handle this part, data cleaning is done. It involves handling of
missing data, noisy data etc.

 (a). Missing Data:

This situation arises when some data is missing in the
data. It can be handled in various ways.
Some of them are:
 1. Ignore the tuples:
This approach is suitable only when the dataset we
have is quite large and multiple values are missing
within a tuple.
2. Fill the Missing values:
There are various ways to do this task. You can
choose to fill the missing values.
 (b). Noisy Data:
Noisy data is a meaningless data that can’t be interpreted
by [Link] can be generated due to faulty data
collection, data entry errors etc. It can be handled in
following ways :
1. Binning Method:
This method works on sorted data in order to
smooth it. The whole data is divided into segments
of equal size and then various methods are
performed to complete the task. Each segmented is
handled separately. One can replace all data in a
segment by its mean or boundary values can be used
to complete the task.

2. Regression:
Here data can be made smooth by fitting it to a
regression [Link] regression used may be
linear (having one independent variable) or multiple
(having multiple independent variables).

3. Clustering:
This approach groups the similar data in a cluster.
The outliers may be undetected or it will fall outside
the clusters.
2. Data Transformation:
This step is taken in order to transform the data in appropriate
forms suitable for mining process. This involves following
ways:
1. Normalization:
It is done in order to scale the data values in a specified
range (-1.0 to 1.0 or 0.0 to 1.0)

2. Attribute Selection:
In this strategy, new attributes are constructed from the
given set of attributes to help the mining process.

3. Discretization:
This is done to replace the raw values of numeric
attribute by interval levels or conceptual levels.

4. Concept Hierarchy Generation:

Here attributes are converted from lower level to higher
level in hierarchy. For Example-The attribute “city” can
be converted to “country”.
3. Data Reduction:
Since data mining is a technique that is used to handle huge
amount of data. While working with huge volume of data,
analysis became harder in such cases. In order to get rid of
this, we uses data reduction technique. It aims to increase the
storage efficiency and reduce data storage and analysis costs.
1. Data Cube Aggregation:
Aggregation operation is applied to data for the
construction of the data cube.

2. Attribute Subset Selection:

The highly relevant attributes should be used, rest all can
be discarded. For performing attribute selection, one can
 use level of significance and p- value of the [Link]
attribute having p-value greater than significance level
can be discarded.

3. Numerosity Reduction:
This enable to store the model of data instead of whole
data, for example: Regression Models.

4. Dimensionality Reduction:
This reduce the size of data by encoding [Link]
can be lossy or lossless. If after reconstruction from
compressed data, original data can be retrieved, such
reduction are called lossless reduction else it is called
lossy reduction. The two effective methods of
dimensionality reduction are:Wavelet transforms and
PCA (Principal Component Analysis).
Experiment 3:Implementing Apriori Algorithm
Apriori Algorithm is a Machine Learning algorithm utilized to
understand the patterns of relationships among the various
products involved. The most popular use of the algorithm is
to suggest products based on the items already in the user's
shopping cart. Walmart specifically has utilized the algorithm
in recommending items to its users.

Step 1: Import the required libraries

1. import numpy as np
2. import pandas as pd
3. from mlxtend.frequent_patterns import apriori, associatio
n_rules
Step 2: Load and explore the data

1. # Now, we will load the Data

2. data1 = pnd.read_excel('Online_Retail.xlsx')
3. [Link]()
4. Output:
InvoiceNo StockCode Description Quantity InvoiceDate UnitPrice CustomerID Country

WHITE
HANGING 2010-12-01 United
0 536365 85123A 6 2.55 17850.0
HEART T-LIGHT 08:26:00 Kingdom
HOLDER

WHITE METAL 2010-12-01 United

1 536365 71053 6 3.39 17850.0
LANTERN 08:26:00 Kingdom

CREAM CUPID
2010-12-01 United
2 536365 84406B HEARTS COAT 8 2.75 17850.0
08:26:00 Kingdom
HANGER
 KNITTED UNION
2010-12-01 United
3 536365 84029G FLAG HOT 6 3.39 17850.0
08:26:00 Kingdom
WATER BOTTLE

RED WOOLLY
2010-12-01 United
4 536365 84029E HOTTIE WHITE 6 3.39 17850.0
08:26:00 Kingdom
HEART.

Step 3: Clean the Data

1. # here, we will strip the extra spaces in the description
2. data1['Description'] = data1['Description'].[Link]()
3.
4. # Now, drop the rows which does not have any invoice n
umber
5. [Link](axis = 0, subset = ['InvoiceNo'], inplace = T
rue)
6. data1['InvoiceNo'] = data1['InvoiceNo'].astype('str')
7.
8. # Now, we will drop all transactions which were done on
credit
9. data1 = data1[~data1['InvoiceNo'].[Link]('C')]
Step 4: Split the data according to the region of
transaction
1. # Transactions done in France
2. basket1_France = (data1[data1['Country'] == "France"]
3. .groupby(['InvoiceNo', 'Description'])['Quantity']
4. .sum().unstack().reset_index().fillna(0)
5. .set_index('InvoiceNo'))
6. # Transactions done in the United Kingdom
7. basket1_UK = (data1[data1['Country'] == "United Kingd
om"]
8. .groupby(['InvoiceNo', 'Description'])['Quantity']
9. .sum().unstack().reset_index().fillna(0)
10. .set_index('InvoiceNo'))
11.
12. # Transactions done in Portugal
13. basket1_Por = (data1[data1['Country'] == "Portugal
"]
14. .groupby(['InvoiceNo', 'Description'])['Quantity
']
15. .sum().unstack().reset_index().fillna(0)
16. .set_index('InvoiceNo'))
17.
18. basket1_Sweden = (data1[data1['Country'] == "Swe
den"]
19. .groupby(['InvoiceNo', 'Description'])['Quantity
']
20. .sum().unstack().reset_index().fillna(0)
21. .set_index('InvoiceNo'))
Experiment 4:Decision Tree Classification
As a marketing manager, you want a set of customers who
are most likely to purchase your product. This is how you can
save your marketing budget by finding your audience. As a
loan manager, you need to identify risky loan applications to
achieve a lower loan default rate. This process of classifying
customers into a group of potential and non-potential
customers or safe or risky loan applications is known as a
classification problem.

Decision Tree Algorithm

A decision tree is a flowchart-like tree structure where an
internal node represents feature(or attribute), the branch
represents a decision rule, and each leaf node represents the
outcome.
The topmost node in a decision tree is known as the root node.
It learns to partition on the basis of the attribute value. It
partitions the tree in recursively manner call recursive
partitioning. This flowchart-like structure helps you in
decision making. It's visualization like a flowchart diagram
which easily mimics the human level thinking. That is why
decision trees are easy to understand and interpret.
How does the Decision Tree Algorithm Work?
The basic idea behind any decision tree algorithm is as
follows:
1. Select the best attribute using Attribute Selection
Measures (ASM) to split the records.
 2. Make that attribute a decision node and breaks the
dataset into smaller subsets.
3. Start tree building by repeating this process recursively
for each child until one of the condition will match:
o All the tuples belong to the same attribute value.

o There are no more remaining attributes.

o There are no more instances.

Decision Tree Terminologies

Root Node: Root node is from where the decision tree starts.
It represents the entire dataset, which further gets divided into
two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree
cannot be segregated further after getting a leaf node.
Splitting: Splitting is the process of dividing the decision
node/root node into sub-nodes according to the given
conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted
branches from the tree.

Parent/Child node: The root node of the tree is called the

parent node, and other nodes are called the child nodes.
How does the Decision Tree algorithm Work?
In a decision tree, for predicting the class of the given dataset,
the algorithm starts from the root node of the tree. This
algorithm compares the values of root attribute with the
record (real dataset) attribute and, based on the comparison,
follows the branch and jumps to the next node.
For the next node, the algorithm again compares the attribute
value with the other sub-nodes and move further. It continues
the process until it reaches the leaf node of the tree. The
complete process can be better understood using the below
algorithm:
 Step-1: Begin the tree with the root node, says S, which
contains the complete dataset.
 Step-2: Find the best attribute in the dataset using
Attribute Selection Measure (ASM).
 Step-3: Divide the S into subsets that contains possible
values for the best attributes.
 Step-4: Generate the decision tree node, which contains
the best attribute.
 Step-5: Recursively make new decision trees using the
subsets of the dataset created in step -3. Continue this
process until a stage is reached where you cannot further
classify the nodes and called the final node as a leaf
node.
Example: Suppose there is a candidate who has a job offer
and wants to decide whether he should accept the offer or Not.
So, to solve this problem, the decision tree starts with the root
node (Salary attribute by ASM). The root node splits further
into the next decision node (distance from the office) and one
leaf node based on the corresponding labels. The next
decision node further gets split into one decision node (Cab
facility) and one leaf node. Finally, the decision node splits
into two leaf nodes (Accepted offers and Declined offer).
Consider the below diagram:

Advantages of the Decision Tree

 It is simple to understand as it follows the same process
which a human follow while making any decision in
real-life.
 It can be very useful for solving decision-related
problems.
 It helps to think about all the possible outcomes for a
problem.
 There is less requirement of data cleaning compared to
other algorithms.
Disadvantages of the Decision Tree
 The decision tree contains lots of layers, which makes it
complex.
 It may have an overfitting issue, which can be resolved
using the Random Forest algorithm.
 For more class labels, the computational complexity of
the decision tree may increase.
Experiment 5: Implementation k-mean clustering
Introduction to K-Means Algorithm
K-means clustering algorithm computes the centroids and
iterates until we it finds optimal centroid. It assumes that the
number of clusters are already known. It is also called flat
clustering algorithm. The number of clusters identified from
data by algorithm is represented by ‘K’ in K-means.
In this algorithm, the data points are assigned to a cluster in
such a manner that the sum of the squared distance between
the data points and centroid would be minimum. It is to be
understood that less variation within the clusters will lead to
more similar data points within same cluster.
Working of K-Means Algorithm
We can understand the working of K-Means clustering
algorithm with the help of following steps −
 Step 1 − First, we need to specify the number of clusters,
K, need to be generated by this algorithm.
 Step 2 − Next, randomly select K data points and assign
each data point to a cluster. In simple words, classify the
data based on the number of data points.
 Step 3 − Now it will compute the cluster centroids.
 Step 4 − Next, keep iterating the following until we find
optimal centroid which is the assignment of data points
to the clusters that are not changing any more −
4.1 − First, the sum of squared distance between data points
and centroids would be computed.
4.2 − Now, we have to assign each data point to the cluster
that is closer than other cluster (centroid).
4.3 − At last compute the centroids for the clusters by taking
the average of all data points of that cluster.
K-means follows Expectation-Maximization approach to
solve the problem. The Expectation-step is used for assigning
the data points to the closest cluster and the Maximization-
step is used for computing the centroid of each cluster.
While working with K-means algorithm we need to take care
of the following things −
 While working with clustering algorithms including K-
Means, it is recommended to standardize the data
because such algorithms use distance-based
measurement to determine the similarity between data
points.
 Due to the iterative nature of K-Means and random
initialization of centroids, K-Means may stick in a local
optimum and may not converge to global optimum. That
is why it is recommended to use different initializations
of centroids.
Implementation in Python
The following two examples of implementing K-Means
clustering algorithm will help us in its better understanding −
Example 1

It is a simple example to understand how k-means works. In

this example, we are going to first generate 2D dataset
containing 4 different blobs and after that will apply k-means
algorithm to see the result.
First, we will start by importing the necessary packages −
%matplotlib inline
import [Link] as plt
import seaborn as sns; [Link]()
import numpy as np
from [Link] import KMeans
The following code will generate the 2D, containing four
blobs −
from [Link].samples_generator
import make_blobs
X, y_true = make_blobs(n_samples=400,
centers=4, cluster_std=0.60,
random_state=0)
Next, the following code will help us to visualize the dataset −
[Link](X[:, 0], X[:, 1], s=20);
[Link]()

Next, make an object of KMeans along with providing

number of clusters, train the model and do the prediction as
follows −
kmeans = KMeans(n_clusters=4)
[Link](X)
y_kmeans = [Link](X)
Now, with the help of following code we can plot and
visualize the cluster’s centers picked by k-means Python
estimator −
[Link](X[:,0], X[:,1], c=y_kmeans,
s=20, cmap='summer')
centers = kmeans.cluster_centers_
[Link](centers[:,0], centers[:,1],
c='blue', s=100, alpha=0.9);
[Link]()