![Page 1: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/1.jpg)
Getting a CLUE: A Method for Explaining Uncertainty Estimates
ML-IRL Workshop at ICLR 2020
Javier Antorán, Umang Bhatt, Tameem Adel,
Adrian Weller, and José Miguel Hernández-Lobato
![Page 2: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/2.jpg)
About Us
Javier Antorán [email protected]
Umang Bhatt [email protected]
José Miguel Hernández-Lobato [email protected]
Adrian Weller [email protected]
Tameem Adel [email protected]
![Page 3: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/3.jpg)
Uncertainty in Predictive Models
°5 0 5
°5
0
5
°10 0 10°10
0
10
°10 0 10°10
0
10
0.0
0.7
H(n
ats)
Quantify Uncertainty through Entropy (Classification) or Variance (Regression)
Have we observed enough data to make confident predictions?
Is there class overlap in our data?
![Page 4: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/4.jpg)
Motivation: Transparency in Deep Learning via Uncertainty
x*Accept a Certain
Prediction
Reject an Uncertain Prediction ?
Get Explanation
ML Practitioner Workflow:
• LIME [Ribeiro et al., 2017]
• SHAP [Lundberg et al., 2017]
• Integrated Gradients [Sundararajan et al., 2017]
• FIDO [Chang et al., 2017]
![Page 5: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/5.jpg)
Related Work: Uncertainty Sensitivity Analysis
Use gradients of predictive uncertainty w.r.t. inputs
[Depeweg et. al., 2017]
![Page 6: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/6.jpg)
Fixing Sensitivity Analysis
What if we could constrain our explanations to the data manifold?
Sensitivity can produce meaningless explanations in high dimensions
![Page 7: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/7.jpg)
Getting a CLUE
Use generative model as proxy for the data manifold:
Counterfactual Latent Uncertainty Explanations
“What is the smallest change we need
to make to an input such that our model
produces more certain predictions”
![Page 8: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/8.jpg)
Getting a CLUE (cont.)
![Page 9: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/9.jpg)
Showing CLUEs to Users
MNIST COMPAS
![Page 10: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/10.jpg)
Comparing CLUE and Sensitivity
fm =ΔH
Δx
![Page 11: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/11.jpg)
A Small User Study on COMPAS and LSAT
![Page 12: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/12.jpg)
A Small User Study on COMPAS and LSAT
Users are able to predict if a model will be uncertain on new examples more accurately when using CLUE than when shown uncertainty estimates.
Is CLUE more helpful than just showing uncertainty estimates?
![Page 13: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/13.jpg)
A Small User Study on MNIST
We modify the MNIST train set to introduce Out Of Distribution uncertainty.
Method N. participants Accuracy
Unc. 5 0.67
CLUE 5 0.88
![Page 14: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/14.jpg)
Summary
• Predictive Uncertainty makes ML systems safer and more reliable
• Sensitivity is not enough to explain Predictive Uncertainty in BNNs
• We introduce CLUE, a method to answer the question:
“How should we change an input such that our model produces more certain predictions?”
• CLUE produces in-distribution explanations which trade-off the amount of
change made to inputs and the amount of uncertainty explained away.
• A small user study finds that CLUEs help users understand the sources of a
model’s uncertainty.
![Page 15: Getting a CLUE: A Method for Explaining Uncertainty Estimates...Getting a CLUE: A Method for Explaining Uncertainty Estimates ML-IRL Workshop at ICLR 2020 Javier Antorán, Umang Bhatt,](https://reader035.vdocument.in/reader035/viewer/2022071604/613f8cc8f0f55d448e4cddfe/html5/thumbnails/15.jpg)
References
• [Antorán et. al., 2020] “Getting a CLUE: A Method for Explaining Uncertainty Estimates”
• [Depeweg et. al., 2017] “Sensitivity Analysis for Predictive Uncertainty in Bayesian Neural Networks”
• [Lundberg et al., 2017] “A Unified Approach to Interpreting Model Predictions”
• [Ribeiro et al., 2017] “"Why Should I Trust You?": Explaining the Predictions of Any Classifier”
• [Sundararajan et al., 2017] “Axiomatic Attribution for Deep Networks”
• [Chang et al., 2017] “Explaining Image Classifiers by Counterfactual Generation”