Scribd
Upload
4 views31 pages

Intelligible GAMs for Healthcare Insights

Uploaded by

vaibhavkirar459
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views31 pages

Intelligible GAMs for Healthcare Insights

Uploaded by

vaibhavkirar459
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Generalized Additive Models

Intelligible Models for


HealthCare
Caruana et. al.
Contributions

 Two case studies where Generalized


Additive Models (intelligible) yield state-
of-the-art accuracy.
 Pneumonia risk prediction
 30-day hospital readmission

 Claim: GAMs is a class of models that can


handle interpretability/accuracy trade-off
quite well

3
Roadmap

 Motivation

 Intelligible Models

 Case study: Pneumonia risk

 Case study: 30 day readmission

4
Roadmap

 Motivation

 Intelligible Models

 Case study: Pneumonia risk

 Case study: 30 day readmission

5
Motivation

 A large project to evaluate application of


ML to healthcare problems
 Predicting probability of death (POD) for
pneumonia patients
 Most accurate models: neural nets (0.86 AUC)
 Logistic regression: 0.77

 Logistic regression was used instead.


Why?

6
Motivation

 Rule based learning method was also


used
 Insight: HasAsthma(x)  LowerRisk(x)
 Counterintuitive?

 Rule based system was intelligible making


it easy to recognize and remove
dangerous rules

 Lack of intelligibility made it harder to


deploy neural nets because it was difficult
to know other problems with the model 7
Motivation

 Many more models but equally


unintelligible today
 SVMs, random forests, boosted trees

 GAMs are both intelligible and accurate!


 Editable by experts

8
Roadmap

 Motivation

 Intelligible Models: GAMs and GA 2MS

 Case study: Pneumonia risk

 Case study: 30 day readmission

9
GAMs

10
GAMs

11
GAMs and GA2Ms

g is a link function: identity (additive model e.g., regression);


log (E[y] / 1 – E[y]) (generalized additive model e.g.,
classification)

fj is a shape function

12
Intelligibility and Accuracy

Lou et. al., 2012


13
Shape Functions

 Regression Splines

 Trees

 Ensembles of Trees

14
GAMs and GA2Ms

 Learning:
 Represent each component as a spline
 Least squares formulation; Optimization problem to
balance smoothness and empirical error
 Regression trees on a single/pair of features
 Gradient boosting with bagging of shallow trees

 GA2Ms: Build GAM first and and then detect


and rank all possible pairs of interactions in
the residual
 Choose top k pairs
 k determined by CV
15
Roadmap

 Motivation

 Intelligible Models

 Case study: Pneumonia risk

 Case study: 30 day readmission

17
Pneumonia Risk
 14,199 pneumonia patients

 train set: 9847

 test set: 4352

 46 features

 Predict POD

 10.86% patients died from pneumonia (1542)


18
Pneumonia Risk: Features

19
Pneumonia Risk: AUC

20
Understanding Outputs of GA2M

Blood Urea Nitrogen

Normal value: 10 to 20
0 means not ordered

21
Understanding Outputs of GA2M

Childhood cancers are associated with high risk of death

22
Roadmap

 Motivation

 Intelligible Models

 Case study: Pneumonia risk

 Case study: 30 day readmission

23
30 day readmission

 195K patients in train, 100K patients in test

 3956 features

 Predict which patients are likely to be


readmitted within 30 days of being released

 Hospitals with high readmission rates are


penalized financially
 Did not provide adequate care earlier

 8.91% of patients readmitted within 30 days


24
AUC

25
Patient level insights

1. Lots of admissions
2. Received lot of
Amoxycilin (strep/pneumonia)
3. Verapamil (hypertension)

p(risk) = 0.9326

26
Patient level insights

1. Lots of admissions 1. Post menopausal


2. Received lot of 2. Cancers that
Amoxycilin (strep/pneumonia) respond well to treatmen
3. Verapamil (hypertension) 3. Not hospitalized much

p(risk) = 0.9326 p(risk) = 0.0873

27
Roadmap

 Motivation

 Intelligible Models

 Case study: Pneumonia risk

 Case study: 30 day readmission

28
Modularity

 Bias term + contributions of individual


features + contributions of pairwise
interactions

 This structure helps us clearly understand


the model

30
Sorting Terms By Importance

 For each patient, we can compute which


term is resulting in what risk score

 Rank terms based on the values of risk


score

 This ranking tells us which features are


contributing to the risk of each patient

31
Feature Shaping vs. Expert
Discretization
 Instead of GAMs learning function shapes,
experts could also provide inputs by
discretizing features

 Expert discretized features were used for


logistic regression model

 However, GAMs outperformed LR


indicating that feature shaping is valuable

32
Correlation != Causation

 GAMs and GA2Ms are intelligible

 But, they are not causal

 What we see in plots are associations


captures from the data but are not causal
implications

 It is often easy to confuse intelligibility of


predictive models with causality
 Please don’t make that mistake! 33

You might also like