Scribd
Upload
23 views3 pages

Logistic Regression Numerical Exercises

The document outlines various calculations and concepts related to logistic regression, including sigmoid function output, log-loss, gradient descent, odds and log-odds, evaluation metrics (precision, recall, F1-score), coefficient interpretation, and regularization. It provides specific examples and asks for calculations based on given data for each topic. The document serves as a comprehensive guide for understanding and applying logistic regression techniques.

Uploaded by

Akash Bartwal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
23 views3 pages

Logistic Regression Numerical Exercises

The document outlines various calculations and concepts related to logistic regression, including sigmoid function output, log-loss, gradient descent, odds and log-odds, evaluation metrics (precision, recall, F1-score), coefficient interpretation, and regularization. It provides specific examples and asks for calculations based on given data for each topic. The document serves as a comprehensive guide for understanding and applying logistic regression techniques.

Uploaded by

Akash Bartwal
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

1.

Sigmoid Function Calculation:


For a logistic regression model, the output of the linear function is given by:

z = 3x1 - 2x2 + 0.5

a. Calculate the probability (output of the sigmoid function) when x1 = 2 and x2 =


1.
b. If the decision threshold is 0.5, what will the predicted class be?

2. Log-Loss Calculation:
You are given the following predicted probabilities from a logistic regression model
for a binary classification problem and their corresponding actual classes:

| Actual Class | Predicted Probability |


|--------------|-----------------------|
|1 | 0.9 |
|0 | 0.7 |
|1 | 0.6 |
|0 | 0.3 |

a. Calculate the log-loss for this dataset using the formula for binary classification
log-loss.

3. Gradient Descent for Logistic Regression:


Suppose you are training a logistic regression model with two features. The linear
combination of weights and features is given by:

z = theta_0 + theta_1 x1 + theta_2 x2


The current values of the parameters are theta_0 = 0.2, theta_1 = 0.5, and
theta_2 = -0.4. For a training example where x1 = 2, x2 = 1, and the actual class y
= 1, the output of the sigmoid function is calculated as 0.65.

a. Compute the error term y - y_hat, where y_hat is the predicted probability.
b. Calculate the gradients of the cost function with respect to theta_0, theta_1,
and theta_2.
c. Update the parameters using a learning rate of alpha = 0.01.

4. Odds and Log-Odds:


A logistic regression model predicts that the probability of a certain event is 0.75.
a. What are the odds of the event occurring?
b. Calculate the log-odds (logit) for the event.

5. Evaluation Metrics – Precision, Recall, and F1-Score:


A logistic regression classifier is used to predict whether a customer will buy a
product (1) or not (0). The model makes the following predictions on a test dataset:

| Actual Class | Predicted Class |


|--------------|-----------------|
|1 |1 |
|1 |0 |
|0 |1 |
|0 |0 |
|1 |1 |
|0 |0 |

a. Calculate the precision of the model.


b. Calculate the recall of the model.
c. Calculate the F1-score of the model.
6. Logistic Regression Coefficients Interpretation:
Suppose you are modeling the probability of a person being diabetic (y = 1) based
on their age (x1) and BMI (x2). The logistic regression model is:

a. Interpret the coefficient for age (x1).


b. Calculate the odds of being diabetic for a person who is 50 years old with a BMI
of 30.
c. What is the probability of this person being diabetic?

7. Regularization in Logistic Regression:


You are training a logistic regression model with regularization. The cost function
for logistic regression with L2 regularization (Ridge) is given by:

a. Suppose lambda = 0.1, the regularization term is calculated as 0.2, and the log-
loss without regularization is 0.6. What is the regularized cost J(theta)?
b. How would increasing lambda affect the learned parameters theta_j and the
model’s performance?

Common questions

Powered by AI

The odds are calculated as P(event)/(1-P(event)) = 0.75/0.25 = 3. The log-odds (logit) is log(odds) = log(3) ≈ 1.098612 .

Compute the error term as y - y_hat (1 - 0.65 = 0.35). The gradient of the cost function with respect to theta_0, theta_1, and theta_2 is calculated as: for theta_0: 0.35; for theta_1: 0.35 * x1 = 0.35 * 2 = 0.7; for theta_2: 0.35 * x2 = 0.35 * 1 = 0.35. Parameters updated by subtracting the product of the learning rate (0.01) and the gradients from the current parameters .

First, compute the value of z as z = 3(2) - 2(1) + 0.5 = 6 - 2 + 0.5 = 4.5. The sigmoid function is given by 1 / (1 + exp(-z)). Therefore, the probability is 1 / (1 + exp(-4.5)) ≈ 0.989013057 .

Given model coefficients and feature values, calculate the linear combination to find z, then apply the sigmoid function, 1/(1 + exp(-z)), to find the probability of being diabetic. The specific values for age and BMI were not provided, but they would be plugged into the logistic equation .

The regularized cost J(theta) = 0.6 + 0.1 * 0.2 = 0.62 .

A coefficient in logistic regression represents the change in the log-odds of the outcome for a one-unit increase in the predictor variable, holding all other variables constant. For age in this context, it affects the odds of being diabetic by altering the log-odds proportionally .

Increasing lambda in L2 regularization penalizes larger weights, encouraging smaller model parameters which can reduce model complexity, helping to mitigate overfitting but may underfit the data if lambda is too large. This results in a trade-off between bias and variance .

Log-loss is calculated as -1/n * Σ (y_i * log(p_i) + (1 - y_i) * log(1 - p_i)), where n is the number of samples. Plugging in the values: (-1/4) * [(1 * log(0.9) + (0 * log(0.1)) + (0 * log(0.7) + 1 * log(0.3)) + (1 * log(0.6) + 0 * log(0.4))] ≈ 0.361711 .

Precision = TP / (TP + FP) = 2 / (2 + 1) = 0.67, Recall = TP / (TP + FN) = 2 / (2 + 1) = 0.67, F1-score = 2 * (Precision * Recall) / (Precision + Recall) ≈ 0.67. True Positives (TP): 2, False Positives (FP): 1, False Negatives (FN): 1 .

If the predicted probability (0.65) is greater than the decision threshold (0.5), the predicted class is 1 .

You might also like