Scribd
Upload
19 views2 pages

SVM Classifier Analysis and Results

This document provides instructions for an assignment on support vector machines. Students are asked to: 1) Generate random training data divided into 3 classes and fit an SVM classifier with a linear kernel. 2) Perform 10-fold cross-validation on the training data to select the optimal cost value. 3) Use the optimal model to make predictions on randomly labeled test data and calculate misclassified observations. 4) Repeat the analysis using a radial basis kernel and select the optimal cost and gamma parameters.

Uploaded by

Austin Azenga
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views2 pages

SVM Classifier Analysis and Results

This document provides instructions for an assignment on support vector machines. Students are asked to: 1) Generate random training data divided into 3 classes and fit an SVM classifier with a linear kernel. 2) Perform 10-fold cross-validation on the training data to select the optimal cost value. 3) Use the optimal model to make predictions on randomly labeled test data and calculate misclassified observations. 4) Repeat the analysis using a radial basis kernel and select the optimal cost and gamma parameters.

Uploaded by

Austin Azenga
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

School of Mathematics and Statistics

MAST90083: Computational Statistics and Data Science


Assignment 3
Weight: 15%

Instructions
Use of any function or library other than what is mentioned in this assignment is not rec-
ommended. Use library e1071 that contains the svm function for this assignment. Unless
specified otherwise, set the seed to 50 for all instances i.e. whenever the random number
generator is invoked by any function you should use a seed. You should also note that due
to the way in which the plotting function is implemented in the library e1071 the decision
boundary for linear kernel case might look jagged. You may use ”rep” and ”sample” function
in addition to the functions that have already been mentioned in the assignment.

Question: Support Vector Machines


1. We are going to produce a random data of size 100 × 2 for each of the three classes
(C=3). This can be generated as an aggregated random data x of size N × 2 as
”matrix(rnorm(N *2), ncol=2)”, where N = 300. Each 100 entries in this matrix
belong to a separate class, first 100 to class 1, next 100 to class 2 and last 100 to
class 3, however since all observations were generated from the same distribution it is
not possible to differentiate among them. To make these 300 entries distinctive and
divide the data into 3 different classes, lets define class specific means in variable z as
”matrix(c(0,0,3,0,3,0),C,2)”. Also, generate a response vector y of size N that contains
labels (1 to 3) for the data in x. Using z and y, assign class specific means to data
points of each class and this operation will change the entries of the matrix x and divide
it into three classes. Use ggplot from the library ”ggplot2” to plot x as a data frame
while using y as a factor for colour assignment. (3 marks)

2. Construct the data frame for the training data as ”tdata=[Link](x = x, y=[Link](y))”
and fit the support vector classifier using svm function by setting the kernel as linear,
and cost as 10 and store the result in svmfit. Now, plot the results as ”plot(svmfit,
tdata)”. Also generate summary using the object svmfit and answer how many support
vectors were there in each class? (1 marks)

3. Using the training data from the previous question, perform a ten-fold cross-validation
by utilizing the function ”tune” and providing it with a list of cost values as 0.001, 0.01,
0.1, 1, 5, 10, 100. Use summary on the object returned by the tune function to find out
at what value of cost, the minimum cross validation error rate was found. For this best
cost value, did the number of support vectors increase? How many support vectors

1
were there in each class? Also, save the best model returned by the tune function as
”bestmod”. (2 marks)

4. Set the seed to 100 and generate a test data following the exact approach of question
1 and the syntax ”testdata=[Link](x=xtest , y=[Link] (ytest))”, the only dif-
ference however is that ytest is now labeled randomly with replacement and not in a
sequence of first 100 to class 1 (label 1) and so on. Now, use predict function with input
arguments as ”bestmod” (from previous question) and ”testdata” to predict the class
label of these test observations and store the results in yp . Use the function ”table”
to print the results in form of a table for the vector of predicted labels (yp ) against
the test labels ytest. How many observations are misclassified? Why in one case the
number of correctly classified observations are greater than 100? (2 marks)

5. Initially, for training, cost and gamma are both set to 1 and then for the tuning purpose
their values are set to 0.1, 1, 10, 100, 1000 and 0.5, 1, 2, 3, 4, respectively. Find how many
observations are misclassified using the best model when the kernel is radial (i.e. repeat
question 1 to 4 with radial kernel). Does the result imply that data is linearly separable
and we do not need the radial kernel? What were the optimal (best) cost and gamma
(parameter of the radial basis function) in this case? (2 marks)

Common questions

Powered by AI

Starting with a linear kernel simplifies initial assessments of class separability, assuming linear boundaries, which is computationally less complex. If linear kernel results are suboptimal, experimenting with radial kernels can reveal hidden non-linear patterns in the data, offering a more complex decision boundary that may better capture inter-class relations .

In the assignment, 'ggplot2' is used to plot the data frame created from matrix 'x' and response vector 'y', where 'y' acts as a factor for color assignment. This visual differentiation aids in assessing class segregation, providing intuitive insights into how effectively the data classes are separated before and after model fitting .

The assignment explores kernel differences by using radial kernels in addition to linear ones. Radial kernel experimentation, involving tuning of 'cost' and 'gamma' parameters, helps determine data separability. The results show whether data inherent separability is linear or requires the radial kernel for better classification, providing insight into the appropriate model complexity .

The 'tune' function in cross-validation assesses different parameter configurations, such as cost values, to find the model with the lowest cross-validation error. This approach offers insights into model robustness, indicating not just a single optimal parameter set, but also highlighting the stability range of acceptable parameters under varied data conditions .

The assignment employs a ten-fold cross-validation using the 'tune' function with specified cost values to determine the best SVM model parameters. It is found that the minimum cross-validation error rate occurs at a specific cost value, and this process allows assessing the number of support vectors for each class under the optimal cost condition .

Setting the seed ensures that the random number generation is consistent across different runs of the assignment. This is critical for reproducibility, enabling the same results to be obtained each time the analysis is performed. It helps maintain consistency in data generation, model training, and evaluation processes .

The assignment involves generating test data, this time with labels assigned randomly. The 'predict' function is used with the best SVM model, and the results are compared with test labels using the 'table' function. The task is to identify misclassifications and draw conclusions about why some classes may have more correctly classified observations than others, despite randomness in labeling .

The assignment requires generating random data of size 100x2 for each of the three classes. This data is aggregated into a matrix 'x' of size 300x2, with each 100 entries corresponding to a class, but initially indistinguishable due to being drawn from the same distribution. To differentiate the data, class-specific means are defined in variable 'z' and a response vector 'y' is created to contain labels for 'x'. This setup allows assigning distinct class-specific means to data points, effectively dividing the data into three classes .

Using a predefined set of cost and gamma values allows systematic exploration of the parameter space for non-linear kernels, like the radial basis function. This approach helps identify the combination that minimizes error, influencing model generalization capabilities. However, it assumes the optimal values lie within the set, potentially missing better solutions outside it .

The jagged appearance of the decision boundary with a linear kernel might result from the implementation specifics of the plotting function within the 'e1071' library, as noted in the assignment. This indicates that while the decision boundary should ideally be smooth, visual artifacts can occur, potentially misleading interpretations if not understood correctly .

You might also like