Scribd
Upload
8 views4 pages

AdaBoost Algorithm Example

The document describes a health-insurance risk dataset with numeric and categorical features, focusing on the prediction of illness based on factors like age, smoking, income, and gender. It details the AdaBoost algorithm's application to the dataset, showcasing the process of selecting weak learners, calculating errors, and updating weights to improve predictions. Ultimately, it concludes that a simple income threshold can perfectly classify illness in the provided examples.

Uploaded by

vennira8880
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views4 pages

AdaBoost Algorithm Example

The document describes a health-insurance risk dataset with numeric and categorical features, focusing on the prediction of illness based on factors like age, smoking, income, and gender. It details the AdaBoost algorithm's application to the dataset, showcasing the process of selecting weak learners, calculating errors, and updating weights to improve predictions. Ultimately, it concludes that a simple income threshold can perfectly classify illness in the provided examples.

Uploaded by

vennira8880
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Health-insurance risk dataset with 3 numeric columns + 1 categorical, and the target

Illness (Yes/No).

Example-1
Row Gender Age Income Smoking Illness Sample
No. (₹/year) (cigs/day) Weights
1 Male 38 420000 0 No 1/5
2 Female 52 360000 5 Yes 1/5
3 Male 45 780000 0 No 1/5
4 Female 29 300000 12 Yes 1/5
5 Male 61 500000 8 Yes 1/5

• Illness risk tends to increase with age


• Smoking is a strong risk factor
• Income varies but is not always directly tied to illness (realistic noise)
• Gender is included as a demographic feature (often present in real datasets)

Example-2
Row No. Gender Age Income (₹/year) BMI Illness Sample Weights
1 Female 33 480000 22.1 No 1/5
2 Male 57 320000 29.5 Yes 1/5
3 Male 41 900000 24.0 No 1/5
4 Female 49 540000 31.2 Yes 1/5
5 Male 36 450000 27.8 No 1/5
AdaBoost solution
1. Select stump-1 (e.g., Smoking ≥ 7 → Yes)
2. Compute error (ε) using sample weights
3. Compute alpha (α)
4. Update weights and normalize
5. Repeat for stump-2 and stump-3
6. Display final weighted voting prediction
Use the standard AdaBoost label encoding:
• Illness Yes = +1
• Illness No = −1
Given data (n = 5)
Initial weights: 𝑤𝑖 = 1/5 = 0.2
Row Gender Age Income Illness y
1 M 41 40000 Yes +1
2 M 54 30000 No −1
3 F 42 25000 No −1
4 F 40 60000 Yes +1
5 M 46 50000 Yes +1

AdaBoost Round 1 (Weak learner 𝒉𝟏 )


Step 1: Choose a simple stump (Gender-based)
Let’s pick a stump that is better than random:
+1 if Gender = Male
ℎ1 (𝑥) = {
−1 if Gender = Female
Predictions
Row Gender y ℎ1 (𝑥) Correct?
1 M +1 +1

2 M −1 +1

3 F −1 −1

4 F +1 −1

5 M +1 +1

Wrong rows = 2 and 4


Step 2: Weighted error 𝜺𝟏
𝜀1 = 𝑤2 + 𝑤4 = 0.2 + 0.2 = 0.4
Step 3: Compute 𝜶𝟏
1 1 − 𝜀1 1 0.6 1
𝛼1 = ln⁡ ( ) = ln⁡ ( ) = ln⁡(1.5) ≈ 0.2027
2 𝜀1 2 0.4 2
Useful values:
• 𝑒 𝛼1 ≈ 1.2247
• 𝑒 −𝛼1 ≈ 0.8165
Step 4: Update weights
AdaBoost update rule:
• If correct: 𝑤𝑖 ← 𝑤𝑖 ⋅ 𝑒 −𝛼1
• If wrong: 𝑤𝑖 ← 𝑤𝑖 ⋅ 𝑒 +𝛼1
Before normalization
• Correct rows (1,3,5): 0.2 × 0.8165 = 0.1633
• Wrong rows (2,4): 0.2 × 1.2247 = 0.2449
Sum:𝑆 = 3(0.1633) + 2(0.2449) = 0.4899 + 0.4898 ≈ 0.9797
Normalize
• Rows 1,3,5: 0.1633/0.9797 ≈ 0.1667
• Rows 2,4: 0.2449/0.9797 ≈ 0.25
So new weights after Round 1:
Row New weight
1 0.1667
2 0.2500
3 0.1667
4 0.2500
5 0.1667
Now “Focus more on rows 2 and 4.”

AdaBoost Round 2 (Weak learner 𝒉𝟐 )


Step 1: Choose best stump using numeric feature (Income)
Try this stump:
+1 if Income ≥ 35000
ℎ2 (𝑥) = {
−1 if Income < 35000
Predictions
Row Income y ℎ2 (𝑥) Correct?
1 40000 +1 +1

2 30000 −1 −1

3 25000 −1 −1

4 60000 +1 +1

5 50000 +1 +1

No mistakes.
Step 2: Weighted error 𝜺𝟐
𝜀2 = 0)
If a weak learner achieves zero error, then:
1 1−𝜀2
• 𝛼2 = 2 ln⁡ ( )becomes infinite
𝜀2

• Practically, the algorithm stops early, because we already have a perfect classifier.
Final model: The income stump alone perfectly predicts Illness for all 5 rows.

Final Answer (Model Rule)


A perfect classifier for this dataset is:
If Income ≥ 35000 → Illness = Yes
Else → Illness = No

You might also like