Explainable AI
Author: Rohit Mande
Founder and CEO, [Link]
Website: [Link]
Website: [Link]
Website: [Link]
Index
• Need of explainable AI
• Why feature importance is not enough?
• LIME
• SHAP
• LIME vs SHAP
Website: [Link]
Website: [Link]
Need of Explainable AI
Website: [Link]
Website: [Link]
Need of Explainable AI
Transparency and Accountability
Understanding how AI models arrive at specific conclusions or predictions is essential for holding systems accountable for their
actions
Building Trust
When users understand the reasoning behind AI decisions, they are more likely to trust and adopt these technologies
Mitigating Bias and Fairness Concerns
By providing insights into the features that influence decisions, stakeholders can assess and address potential biases
Human-AI Collaboration Website: [Link]
When users understand the rationale behind AI suggestions or decisions, they can provide valuable feedback, corrections, or
insights
Website: [Link]
Why feature importance is not enough?
Website: [Link]
Website: [Link]
Why feature importance is not enough?
Disadvantages of global feature importance:
Global feature importance such as feature importance from LR, random forest, xgboost gives
important features based on the complete training data
These methods are not able to give feature importance at the instance level i.e., for every single
row
Global feature importance might not provide consistent explanations for different instances,
leading to difficulties in trust and reliability.
Website: [Link]
Website: [Link]
Model-Agnostic Explanations
Model-Agnostic Explanations:
These methods can be applied to any machine learning model. They focus on understanding the
behavior of the model without relying on its internal structure.
Designed to provide local interpretability by explaining the predictions of a model for a specific
instance.
These methods consider feature interactions by providing locally faithful explanations.
Explainable AI methods:
LIME
SHAP Website: [Link]
Website: [Link]
LIME
Website: [Link]
Website: [Link]
LIME
LIME: Local Interpretable Model-agnostic Explanations
LIME is designed to provide insights into individual predictions rather than global model behavior
Image source: LIME research paper
Website: [Link]
Website: [Link]
LIME Steps
Create the perturbed instances near the instance for
which prediction is to be explained
Perturbed instances are given to ML model to get the
predicted label
Perturbed Instances which are closer to the current
instance are given higher weightage
Linear interpretable model is learned from the weighted
samples, and it provides explanations for a particular
prediction for the instance
Image source: LIME research paper
Model's coefficients can be inspected and seen as
proxies for local feature importance.
Website: [Link]
Website: [Link]
LIME
LIME builds Linear Regression Model to create a locally explainable model in case of regression
LIME builds Logistic Regression Model to create a locally explainable model in case of
classification
Coefficients of the features can be used for explaining the model for that instance
Website: [Link]
Website: [Link]
SHAP
Website: [Link]
Website: [Link]
SHAP
SHAP: SHapley Additive exPlanations
SHAP can provide local as well as global feature importance
SHAP values are based on cooperative game theory, specifically Shapley values, which fairly
distribute the "payoff" among a group of contributors.
Website: [Link]
Website: [Link]
How SHAP works?
Generate All Possible Feature Combinations
Enumerate all possible subsets (coalitions) of features, excluding and including each feature
individually. For a dataset with three features (A, B, C), you would generate coalitions like {}, {A}, {B},
{C}, {A, B}, {A, C}, {B, C}, {A, B, C}.
Evaluate Model Predictions for Each Coalition (combination)
For each subset of features, evaluate the model's prediction on the dataset. This includes the empty
coalition {}.
Website: [Link]
Website: [Link]
How SHAP works?
Calculate Marginal Contributions:
For each subset (coalition), calculate the marginal contribution of each feature that is not in the
coalition but is added to it.
Marginal Contribution(i,S) = Model Prediction(S ∪ {i}) − Model Prediction(S)
Average Marginal Contributions:
For each feature, average its marginal contributions over all possible coalitions.
This average reflects the feature's Shapley value, representing its fair contribution to the model's
prediction.
Website: [Link]
Website: [Link]
SHAP Example
Website: [Link]
Website: [Link]
LIME vs SHAP
Website: [Link]
Website: [Link]
LIME vs SHAP
Go for LIME when Simplicity and Speed Matter
LIME tends to be computationally faster compared to SHAP, especially when dealing with complex
models or large datasets
SHAP is well-suited for scenarios where you require both local and global interpretability.
SHAP values inherently account for interactions between features.
SHAP values are designed to be consistent across different instances, providing a more stable and
Website:
reliable framework for interpreting [Link]
model predictions across a range of data points.
Website: [Link]
Questions?
Website: [Link]
Website: [Link]