crime prevention - feedzai · the fictional example of tyler and xmusic: the false positive rate...
TRANSCRIPT
![Page 1: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/1.jpg)
![Page 2: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/2.jpg)
© 2019 Feedzai© 2019 Feedzai
The FATE of Financial Crime PreventionPedro Saleiro
Money2020 - Las Vegas, October 2019
Pedro SaleiroData Science Manager
The FATE ofFinancial Crime Prevention:Fairness, Accountability,Transparency and Ethics in AI
![Page 3: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/3.jpg)
© 2019 Feedzai
The 3 main ethical problems of AI today(not adequately addressed by regulation, yet)
![Page 4: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/4.jpg)
© 2019 Feedzai
1. Bias and Fairness
![Page 5: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/5.jpg)
![Page 6: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/6.jpg)
© 2019 Feedzai
What about the online payments industry?
The fictional example of Tyler and Xmusic:The False Positive Rate for Tyler’s zip code is
5X higher than average
Xmusic is not aware of the biases ofits fraud prevention AI solution
![Page 7: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/7.jpg)
© 2019 Feedzai
2. Transparency
![Page 8: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/8.jpg)
© 2019 Feedzai
State-of-the-Art in Explainable AI
Transaction: 23132Score: 875
Fraud Non-fraud
![Page 9: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/9.jpg)
© 2019 Feedzai
3. Accountability
![Page 10: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/10.jpg)
© 2019 Feedzai
Companies must use AI that is compliantand tied to their values!
But...
Increasing complexity (e.g., deep learning) and automation (e.g., AutoML) makes it hard
to understand AI and its impact.
![Page 11: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/11.jpg)
© 2019 Feedzai
FATE to the rescue!(let’s talk about a few research projects)
![Page 12: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/12.jpg)
© 2019 Feedzai
1. Bias and Fairness Audits
![Page 13: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/13.jpg)
© 2019 Feedzai
What do you need to audit an AI system?
1. Fraud predictions 2. The true outcome
(fraud/not fraud)3. Protected attributes
you care about4. Select bias metrics5. Bias audit toolkit (e.g. Aequitas)
![Page 14: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/14.jpg)
© 2019 Feedzai
The Awareness Effect
![Page 15: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/15.jpg)
© 2019 Feedzai
2. Human-Interpretable Explanations(and adequate evaluation)
![Page 16: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/16.jpg)
© 2019 Feedzai
How a great explanation looks like
Transaction: 23132Model Score: 875
Explanation:Suspicious because it's an
high-speed ordering from a bot
Fraud Non-fraud
![Page 17: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/17.jpg)
© 2019 Feedzai
Semantic Layer for Explanations
Linking low-level model-based explanations with semantic
concepts from a fraud ontology
![Page 18: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/18.jpg)
© 2019 Feedzai
Systematic Evaluation of Explanations
![Page 19: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/19.jpg)
© 2019 Feedzai
3. Robust and Multi-Objective Model Selection
![Page 20: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/20.jpg)
© 2019 Feedzai
The Awareness Effect (again!)
![Page 21: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/21.jpg)
© 2019 Feedzai
Multi-Objective Model Selection
From all the compliant models optimize:
1. Overall performance (e.g., Recall)2. Fairness (e.g., FPR disparity nationality)3. Performance Stability4. Quality of Explanations5. Energy consumption (!!)
![Page 22: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/22.jpg)
© 2019 Feedzai
OutlookFairness, Transparency and Accountability arecurrent ethical issues in AI
FATE research at Feedzai is working on:1. Fairness Awareness through audits2. Human-Interpretable and useful explanations3. Robust and multi-objective model selection
We are empowering our clients with control to develop AI that is compliant, tied to their values and function as intended!
![Page 23: Crime Prevention - Feedzai · The fictional example of Tyler and Xmusic: The False Positive Rate for Tyler’s zip code is 5X higher than average ... Robust and multi-objective model](https://reader033.vdocument.in/reader033/viewer/2022041705/5e445cd178a1511848332c62/html5/thumbnails/23.jpg)
© 2019 Feedzai
THANK YOU