College of Engineering Guindy

M o du l e V

Explainable AI using LIME

Presented By:
Geetha

1
 College of Engineering Guindy

Need for Trustworthiness

• World’s first AI safety summit hosted in UK &
Representatives from 28 countries signed the Bletchley
declaration
• Acknowledge AI’s disruptive potential for significant benefits
across sectors.
• Recognition of potential catastrophic consequences
underscores the need for monitoring and regulation.
• Building Responsible AI:
• AI Safety Summit underscores the urgency of developing
safe, trustworthy and responsible AI
• In a data-centric landscape, AI demands privacy
preservation, interpretable insights, and a balance
between responsibility and accuracy.

2
 College of Engineering Guindy

Why Model Understanding?

• Utility • Stakeholders
• Debugging • End users
• Bias detection • Decision makers
• Recourse • Regulatory agencies
• If and when to trust model predictions • Researchers and engineers
• Determine suitability for deployment

• Model understanding is absolutely critical in several domains – particularly

involving high stakes decision

3
 College of Engineering Guindy

Classification: Wolf or a Husky

We have built a Snow Detector

4
 College of Engineering Guindy

Explainable AI

• Interpretable machine learning

models provides techniques to
understand and validate how the
model works.
• Build interpretable models
• Derive explanations for complex models

• Focus of XAI is to build human

interpretable models

5
 College of Engineering Guindy

Categorization in XAI

Explanation
Agnosticity Scope Data Types
Types
Model Image Visual
Global
Agnostic Explanation Feature
Graph
Importance

Text / Speech Data Points

Model Local
Specific Explanation Tabular Surrogate Models

6
 College of Engineering Guindy

Inherently Explainable Models

• Logistic Regression
• Extension of linear regression to classification models
• Output ranges between 0 to 1
• Linear regression learns the coefficients (𝛽) for a weighted sum
of feature inputs
𝑦 = 𝛽0 + 𝛽1 𝑥1 + 𝛽2 𝑥2 + . .
• For logistic regression, the impact of input features as exp(𝛽𝑗 ).

8
 College of Engineering Guindy

Arrival of Big Data

Lots of dimensions Complex Surfaces

Black-box Models

9
 College of Engineering Guindy

Techniques for Model Interpretability

• Some of the techniques and libraries are

• LIME ( Local Interpretable Model-Agnostic Explanations)
• SHAP (Shapley Additive Explanations)
• ELI5 (Explain Like I’m 5)
• SKATER

10
 College of Engineering Guindy

Local Interpretable Model-agnostic Explanations (LIME)

Local Interpretable Model-agnostic Explanations

Explanations Meaningful Any machine Artifacts that

are locally interpretations of learning model provide an
faithful the models can be made understanding
behavior explainable. between input to
Example: text, a ML model and
image, tabular the models
data predictions

Kaggle stroke Dataset

• 67 subjects
• LIME is widely used to explain black-box models (highly non-linear) at a local level • 10 features
• LIME uses simple, explainable model to understand its prediction

11
 College of Engineering Guindy

How does LIME Work ?

• LIME works by zooming into the local

area of the individual prediction and
provide explanations in that local region

12
 College of Engineering Guindy

How does LIME Work ?

• LIME works by zooming into the local

area of the individual prediction and
provide explanations in that local region
• LIME fits the linear interpretable models
in the local region

13
 College of Engineering Guindy

How does LIME Work ?

• LIME works by zooming into the local

area of the individual prediction and
provide explanations in that local region
• LIME fits the linear interpretable models
in the local region
• Local approximation of complex model

14
 College of Engineering Guindy

How to train a surrogate model ?

• Generate perturbed data samples around the data instance
• Calculate distance between permutations and original observations
• Make predictions on new data using complex model
• Fit a simple model to the permuted data
• Feature weights from the simple model make explanations for the complex models local behaviour

Generate
𝑋𝑖 distance Weight classifier Generate
perturbed
Input Instance 𝑁𝑠 samples
kernel samples Score func. prediction

Top k
𝑊𝑖 Learn linear
feature
Explanation model
selection

15
 College of Engineering Guindy

Math in LIME

Good approximation Stay simple

Complex Proximity
Simple
model
interpretable
Family of model
interpretable
models

16
 College of Engineering Guindy

Train a Surrogate Model

• New Dataset for simple linear classifier (𝑔)

• Labels: Prediction of complex model
• Features: Newly generated points

17
 College of Engineering Guindy

Train a Surrogate Model

𝜋𝑥 uses exponential kernel as distance metric, 𝑔 as Sparse linear model

18
 College of Engineering Guindy

How LIME Works

19
 College of Engineering Guindy

Sample Code

Fit a local model for each

data point

20
 College of Engineering Guindy

Sample Code & Results

21
 College of Engineering Guindy

Sample results

22
 College of Engineering Guindy

Summary

• Work on any black-box models

• Model internals are hidden
• Work with any data types
• Using prior knowledge we can validate the explanations and create trust

23
 College of Engineering Guindy

Applications

• Healthcare
• BFSI – Banking, Financial Sector,
Insurance
• Automobiles
• Judicial Systems
• Chatbots
• Robotics

24
 College of Engineering Guindy

References
1. Christoph Molnar, Interpretable Machine Learning, [Link], 2020.
2. Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the
predictions of any classifier." In Proceedings of the 22nd ACM SIGKDD international conference on
knowledge discovery and data mining, pp. 1135-1144. 2016.
3. D. Gaspar, P. Silva and C. Silva, "Explainable AI for Intrusion Detection Systems: LIME and SHAP
Applicability on Multi-Layer Perceptron," in IEEE Access, vol. 12, pp. 30164-30175, 2024, doi:
10.1109/ACCESS.2024.3368377.
4. S. Alwarthan, N. Aslam and I. U. Khan, "An Explainable Model for Identifying At-Risk Student at Higher
Education," in IEEE Access, vol. 10, pp. 107649-107668, 2022, doi: 10.1109/ACCESS.2022.3211070.
5. I. Maouche, L. S. Terrissa, K. Benmohammed and N. Zerhouni, "An Explainable AI Approach for Breast
Cancer Metastasis Prediction Based on Clinicopathological Data," in IEEE Transactions on Biomedical
Engineering, vol. 70, no. 12, pp. 3321-3329, Dec. 2023, doi: 10.1109/TBME.2023.3282840.
6. N. Ranjbar and R. Safabakhsh, "Using Decision Tree as Local Interpretable Model in Autoencoder-
based LIME," 2022 27th International Computer Conference, Computer Society of Iran (CSICC), Tehran,
Iran, Islamic Republic of, 2022, pp. 1-7, doi: 10.1109/CSICC55295.2022.9780503.
7. [Link]
model-interpretation-e8ebe5afc608

25
 College of Engineering Guindy

Thank You

26