Digit Recognition with Machine Learning
Uploaded by
prarit.workDigit Recognition with Machine Learning
Uploaded by
prarit.workCommon questions
Powered by AISupport Vector Machine (SVM) outperformed other models in the digit recognition task because it effectively utilized kernel tricks, specifically the Radial Basis Function (RBF), to find optimal hyperplanes for class separation. This capability allows SVM to handle non-linear relationships and complex patterns in the dataset, leading to an accuracy of over 99.1% . Additionally, the SVM maintained strong performance during cross-validation, indicating stability and robustness across different data subsets .
Logistic Regression achieved a respectable performance with accuracy over 93%, demonstrating that even simple linear models can be effective with well-preprocessed data. However, more complex models like SVM and Random Forest outperformed it significantly (99.1% and 96.1%, respectively) due to their ability to handle non-linear data relationships and complex feature interactions. SVM's kernel application and Random Forest's ensemble approach enhance predictive accuracy and generalization, which Logistic Regression's linear boundaries could not match .
The Support Vector Machine (SVM) and K-Nearest Neighbors (KNN) models both exhibited great stability with the highest cross-validation scores, recorded as 0.9882 for SVM (± 0.0052) and KNN (± 0.0087). This indicates that these models consistently performed well across different subsets of the dataset, showing that they have strong generalization capabilities and are unlikely to be overfitting to noise or specific data structures present in the original training set .
Normalization in the preprocessing phase involves scaling pixel values from 0 to the range of 0 to 1 by dividing by 16. This step is crucial as it ensures that the feature range is consistent across all inputs, helping improve model convergence rates and overall accuracy. In machine learning, normalized data allows algorithms like logistic regression or SVM to function optimally without being skewed by varying scales in feature sets .
The attempt to use Principal Component Analysis (PCA) for dimensionality reduction resulted in a slight drop in model accuracy. The reduction in performance was attributed to the possibility that critical information necessary for accurate classification was lost during PCA transformation, leading the researchers to retain the original normalized features instead .
The train-test split in this study, dividing 80% for training and 20% for testing while ensuring a stratified split, ensures that the model has a representative sample of the data during both phases. Stratification maintains class distributions, making the evaluation reliable and true to real-world scenarios, preventing imbalanced learning or biased performance outcomes that could occur with unequal class representation .
The confusion matrix provides a detailed breakdown of actual versus predicted classes, allowing for the evaluation of model performance beyond simple accuracy metrics. It helps identify specific types of errors, such as false positives or false negatives, and assesses how well a model distinguishes between different classes. This detailed analysis aids in understanding not only where a model excels but also where it consistently misclassifies, offering insights for targeted improvements .
The Decision Tree model exhibited the weakest performance among the models used, with an accuracy of 83.3%, largely due to its simple structure and tendency to overfit small datasets. In contrast, Random Forest, an ensemble of multiple decision trees, achieved a much better accuracy of 96.1%. This improved performance is because Random Forest mitigates overfitting through bootstrapping and averaging multiple trees’ predictions, enhancing generalizability .
K-Nearest Neighbors (KNN) is particularly effective for the UCI Digits dataset because the digit images tend to cluster well in pixel-space, allowing the non-parametric model to classify based on proximity effectively. This clustering makes it easy for KNN to determine the majority class among its k neighbors, leading to high accuracy rates, around 98.6% in this case .
Decision Trees showed a relatively low performance with an accuracy of 83.3% due to their inherent tendency to overfit small dataset samples. Their simple structure, relying on single feature splits, fails to capture complex patterns present in the dataset effectively. Furthermore, without additional predictive trees or ensemble techniques like Random Forests, single Decision Trees lack robustness and are more susceptible to variations in the input data .