Regularization in ML overfitting underfitting Good balance Lasso, Ridge & Elastic NetRegularization in Machine Learning * While developing machine learning models you must have encountered a situation in which the training accuracy of the model is high but the validation accuracy or the testing accuracy is too low. * This is the case which is popularly known as overfitting in the domain of machine learning.Regularization in Machine Learning : What are Overfitting? * Overfitting is a phenomenon that occurs when a Machine Learning model is constrained to the training set and not able to perform well on unseen data. * That is when our model learns the noise in the training data as well. * This is the case when our model memorizes the training data instead of learning the patterns in it.Regularization in Machine Learning What are Underfitting? * Inthe case of the underfitting model is unable to perform well even on the training data hence we cannot expect it to perform well on the validation data. * This is the case when we are supposed to increase the complexity of the model or add more features to the feature set.Regularization in Machine Learning Under-fitting Appropirate-fitting Over-fitting (too simple to (forcefitting-too explain the variance) good to be true)Regularization in Machine Learning What are Bias and Variance? * Bias refers to the errors which occur when we try to fit a statistical model on real- world data which does not fit perfectly well on some mathematical model. * If we use a way too simplistic a model to fit the data then we are more probably igh Bias to learn the patterns in the data at hand and hence performs poorly. face the situation h refers to the case when the model is unable * Variance implies the error value that occurs when we try to make predictions by using data that is not previous} en by the model. ag high variance that occurs when the model learns * There is a situation known noise that is present in the data.Regularization in Machine Learning Price Size Size + Oyx 0+ Oyx + Opx2 + Ord + Bax! High bai: iderfit) High vari: igh bais (underfit) ih varjanceRegularization in Machine Learning * Regularization is a technique used to reduce errors by fitting the function appropriately on the given training set and avoiding overfitting. The commonly used regularization techniques are : * Lasso Regularization — L1 Regularization * Ridge Regularization — L2 Regularization * Elastic Net Regularization — L1 and L2 RegularizationRegularization in Machine Learning Lasso Regression A regression model which uses the L1 Regularization technique is called LASSO (Least Absolute Shrinkage and Selection Operator) regression. Lasso Regression adds the “absolute value of magnitude” of the coefficient as a penalty term to the loss function(L). Lasso regression also helps us achieve feature selection by penalizing the weights to approximately equal to zero if that feature does not serve any purpose in the model.Regularization in Machine Learning Lasso Regression el me =o * where, * m- Number of Features + n- Number of Examples * y,- Actual Target Value * Yj, - Predicted Target ValueRegularization in Machine Learning Ridge Regression * Aregression model that uses the L2 regularization technique is called Ridge regression. * Ridge regression adds the “squared magnitude” of the coefficient as a penalty ea esate term to the loss function(L). Yost = 1 Cost = ¢Regularization in Machine Learning Elastic Net Regression This model is a combination of L1 as well as L2 regularization. That implies that we add the absolute norm of the weights as well as the squared measure of the weights. With the help of an extra hyperparameter that controls the ratio of the L1 and L2 regularization. Cost = 13} 2+ (a) DP, jwil +a D2 w?)