Machine Learning Basics Question Bank
Uploaded by
KavithaMachine Learning Basics Question Bank
Uploaded by
KavithaCommon questions
Powered by AIIn batch learning, the model is trained using the entire training dataset in one go. It is efficient for scenarios where data is static and can be processed all at once. Online learning, on the other hand, updates the model incrementally as new data arrives. This approach is suitable for dynamic environments where data continuously evolves, allowing the model to adapt over time.
The bias-variance trade-off is a critical concept in machine learning that describes the balance between two sources of error leading to model inaccuracies: bias (error due to overly simplistic assumptions in the learning algorithm) and variance (error due to complexity in the model making it sensitive to small fluctuations in the training data). High bias can lead to underfitting, where the model is unable to capture the underlying trend of the data. High variance can lead to overfitting, where the model captures noise as if it were significant signal. The goal is to find a right balance that minimizes total error.
A machine learning model's tendency towards underfitting or overfitting is influenced by its complexity relative to the size of the training dataset. A model with high complexity (many parameters) is more prone to overfitting, especially on small datasets, as it can capture even the noise as patterns. Conversely, a model with low complexity may not capture all significant patterns leading to underfitting. The choice of the model architecture, hyperparameters, and the amount and quality of data are key factors.
Initializing the weight vector to zero in a fully connected neural network can lead to symmetry breaking problems. If all weights are initialized to the same value, then all neurons in a layer will compute the same output and gradients during backpropagation, which means they will be updated equally. This symmetry prevents the network from breaking uniformity and learning useful features. Random initialization helps in breaking this symmetry by giving the neurons the opportunity to learn different features.
A machine learning method is classified as supervised if it involves learning from labeled data, where the training dataset includes input-output pairs. The model learns to predict the output from inputs. Unsupervised learning, in contrast, involves using data without explicit labels, focusing on finding hidden structures or patterns in the input data. The presence or absence of labels in the training data is the key determinant.
High variance in a machine learning model indicates that the model is complex and heavily tailored to the training data. This results in the model capturing noise as if it were a significant underlying pattern, which leads to overfitting. When a model overfits, it performs well on the training set by memorizing the data, but it fails to generalize to unseen data and performs poorly on the test set.
In model-based learning methods, hyperparameters define the model architecture and learning process parameters that aren't improved by the learning algorithm itself. They play a crucial role in shaping the model's learning capability, stability, and convergence characteristics. The choice of hyperparameters, such as learning rate, the number of hidden layers, or dropout rates, significantly affects the model's performance and the trade-off between bias and variance.
The effectiveness of a Support Vector Machine (SVM) primarily depends on the selection of the kernel and the appropriate setting of the hyperparameters like the soft margin parameter. These choices influence the model's ability to handle non-linearly separable data and avoid overfitting by controlling the margin width.
If a decision tree performs poorly on both the training and test datasets, it could be due to underfitting. Potential causes include the tree being too shallow, meaning it has not captured the complexity of the data, incorrect choice of splitting criteria, or insufficient training data leading to inability to capture underlying data patterns effectively.
Numerical instability during optimization can be caused by ill-conditioned optimization landscapes, inappropriate learning rates, or gradients that are too large or too small (vanishing/exploding gradients problem). These issues can be mitigated through techniques like normalization, using more stable optimization algorithms (e.g., Adam optimizer), proper initialization of weights, and adjusting the learning rate dynamically.