Machine Learning Assignment: Week 7 MCQs
Uploaded by
Sovit Kumar PradhanMachine Learning Assignment: Week 7 MCQs
Uploaded by
Sovit Kumar PradhanCommon questions
Powered by AIKey differences include: Bagging, or Bootstrap Aggregating, can be parallelized easily and typically uses sampling with replacement, while boosting is inherently sequential and uses weighted sampling techniques. Bagging usually does not increase the error of a base classifier, while boosting has the potential to increase error if not managed carefully .
The VC-dimension provides a quantitative measure of model complexity in computational learning theory. A higher VC-dimension often implies that a more complex model can fit a wider variety of data patterns. However, it also suggests a higher sample complexity is needed to ensure the model generalizes well and does not overfit the training data. Thus, understanding VC-dimension helps balance model expressiveness with the risk of overfitting .
Base learners in an ensemble model can be selected from the same algorithm with different hyperparameters, from different algorithms entirely, or from different training spaces. These approaches allow for the creation of diverse ensembles that can capture various aspects of the data, improving overall predictive performance .
The VC-dimension of a hypothesis space is a measure of the space's capacity to classify data points. Specifically, it is the maximum number of points that can be shattered (correctly classified in all possible ways) by the hypothesis space. If the VC-dimension is 6, it means there exists at least one set of 6 points that can be perfectly classified, while no set of 7 points can be shattered by the hypothesis space .
Decision trees are not considered an ensemble learning algorithm because they do not aggregate multiple models' predictions to arrive at a final prediction. In pure form, a decision tree stands alone and does not involve combining results from multiple trees or models .
The primary benefits of using an ensemble model are better performance and a more generalized model. Ensemble models combine the predictions of multiple base models to reduce variance and improve predictive accuracy. However, they typically reduce interpretability due to the complexity introduced by multiple models .
Setting a limit on the weight of misclassified points in AdaBoost can help make the final classifier more robust to outliers, as it prevents outlier points from dominating the learning process over iterations. However, this modification may also result in a lower overall performance because it limits the algorithm's ability to focus on difficult, misclassified points .
An ensemble technique might perform poorly if the models lack diversity because the ensemble's advantage comes from combining varied perspectives. High diversity among models ensures that different errors are made by different models, which the ensemble can average out, reducing overall error. If models are too similar, they may make the same mistakes, and the ensemble effect is undermined .
Ensemble methods tend to perform better when the individual base models have less correlation among their predictions. This diversity allows the ensemble to reduce variance and increase accuracy, as errors can be averaged out across the diverse models .
AdaBoost focuses on challenging subsets by adjusting weights, giving higher emphasis to previously misclassified points. Each stage selects a classifier that minimizes the current weighted error, better addressing misclassified points. Classifier weights are also determined by their accuracy, which is based on these weighted errors .