![Page 1: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/1.jpg)
Technical and Societal Critiques of ML
CS 229, Fall 2019
slides by Chris Chute, Taide Ding, and Andrey Kurenkov
November 15, 2019
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 1 / 35
![Page 2: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/2.jpg)
Overview
1 Adversarial examples
2 Interpretability
3 Expense: Data and compute
4 Community weaknesses
5 Ethical Concerns
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 2 / 35
![Page 3: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/3.jpg)
Adversarial examples
Figure: Left: Correctly classified image. Right: classified as Ostrich. Reproducedfrom [1].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 3 / 35
![Page 4: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/4.jpg)
Adversarial examples
Invalid smoothness assumption. “For a small enough radius ε > 0 in thevicinity of a given training input x , an x + r satisfying ‖r‖ < ε will getassigned a high probability of the correct class by the model” [1].
How to construct: [2, 3].
How to defend: [1, 4, 5, 6].
Still an open problem
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 4 / 35
![Page 5: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/5.jpg)
Constructing adversarial examples
Fast gradient sign method [2]. Given input x, add noise η in thedirection of the gradient
xAdv = x + η = x + ε · sign(∇xJ(θ, x , y)).
Figure: FGSM example, GoogLeNet trained on ImageNet, ε = .007. Source: [2].
Intuition: by perturbing the example in the direction of the gradient, youincrease the cost function w.r.t. the correct label most efficiently
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 5 / 35
![Page 6: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/6.jpg)
Constructing adversarial examples
Fast gradient sign method [2]. Given input x, add noise η in thedirection of the gradient
xAdv = x + η = x + ε · sign(∇xJ(θ, x , y)).
Figure: FGSM example, GoogLeNet trained on ImageNet, ε = .007. Source: [2].
Intuition: by perturbing the example in the direction of the gradient, youincrease the cost function w.r.t. the correct label most efficiently
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 5 / 35
![Page 7: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/7.jpg)
Constructing adversarial examples
Fast gradient sign method [2]. Given input x, add noise η in thedirection of the gradient
xAdv = x + η = x + ε · sign(∇xJ(θ, x , y)).
Figure: FGSM example, GoogLeNet trained on ImageNet, ε = .007. Source: [2].
Intuition: by perturbing the example in the direction of the gradient, youincrease the cost function w.r.t. the correct label most efficiently
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 5 / 35
![Page 8: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/8.jpg)
Properties
Change often indistinguishable to human eye.
Adversarial examples generalize across architectures, training sets.
Adversarial perturbations η generalize across examples.
Can construct in the physical world (e.g. stop signs)
Figure: A turtle. Or is it a rifle? Reproduced from [7].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 6 / 35
![Page 9: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/9.jpg)
Properties
Change often indistinguishable to human eye.
Adversarial examples generalize across architectures, training sets.
Adversarial perturbations η generalize across examples.
Can construct in the physical world (e.g. stop signs)
Figure: A turtle. Or is it a rifle? Reproduced from [7].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 6 / 35
![Page 10: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/10.jpg)
Properties
Change often indistinguishable to human eye.
Adversarial examples generalize across architectures, training sets.
Adversarial perturbations η generalize across examples.
Can construct in the physical world (e.g. stop signs)
Figure: A turtle. Or is it a rifle? Reproduced from [7].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 6 / 35
![Page 11: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/11.jpg)
Properties
Change often indistinguishable to human eye.
Adversarial examples generalize across architectures, training sets.
Adversarial perturbations η generalize across examples.
Can construct in the physical world (e.g. stop signs)
Figure: A turtle. Or is it a rifle? Reproduced from [7].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 6 / 35
![Page 12: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/12.jpg)
Defenses
Train on mixture of clean x , perturbed x [1].
Use distillation [4] as a defense [5]. Instead of training with hard(one-hot) labels, train with a high-temperature softmax output ofanother NN trained with hard labels
Many other defenses: [6]. But... Goodfellow et al. [2] claimsfundamental problem with linear models (and high-dimensional input):
wT x = wTx + wTη.
Arms race: generating adversarial examples with GANs (Ermon Lab:[3])
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 7 / 35
![Page 13: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/13.jpg)
Defenses
Train on mixture of clean x , perturbed x [1].
Use distillation [4] as a defense [5]. Instead of training with hard(one-hot) labels, train with a high-temperature softmax output ofanother NN trained with hard labels
Many other defenses: [6]. But... Goodfellow et al. [2] claimsfundamental problem with linear models (and high-dimensional input):
wT x = wTx + wTη.
Arms race: generating adversarial examples with GANs (Ermon Lab:[3])
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 7 / 35
![Page 14: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/14.jpg)
Defenses
Train on mixture of clean x , perturbed x [1].
Use distillation [4] as a defense [5]. Instead of training with hard(one-hot) labels, train with a high-temperature softmax output ofanother NN trained with hard labels
Many other defenses: [6]. But... Goodfellow et al. [2] claimsfundamental problem with linear models (and high-dimensional input):
wT x = wTx + wTη.
Arms race: generating adversarial examples with GANs (Ermon Lab:[3])
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 7 / 35
![Page 15: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/15.jpg)
Defenses
Train on mixture of clean x , perturbed x [1].
Use distillation [4] as a defense [5]. Instead of training with hard(one-hot) labels, train with a high-temperature softmax output ofanother NN trained with hard labels
Many other defenses: [6]. But... Goodfellow et al. [2] claimsfundamental problem with linear models (and high-dimensional input):
wT x = wTx + wTη.
Arms race: generating adversarial examples with GANs (Ermon Lab:[3])
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 7 / 35
![Page 16: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/16.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 17: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/17.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 18: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/18.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 19: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/19.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?
3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 20: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/20.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?
4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 21: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/21.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context
5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 22: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/22.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 23: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/23.jpg)
Interpretability
Switching gears: Interpretability.
Considerations (from Lipton: ”The Mythos of Model Interpretability”[8]):
1 Trust: Costs of relinquishing control - is the model right where humansare right?
2 Causality: Need to uncover causal relationships?3 Transferability: generalizes to other distributions / novel environments?4 Informativeness: not just answer, but context5 Fairness and ethics: Will real-world effect be fair?
Main problem: Evaluation metrics that only look at predictions andground truth labels don’t always capture the above considerations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 8 / 35
![Page 24: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/24.jpg)
Interpretability: Fallacies
Fallacy 1. “Linear models are interpretable. Neural networks areblack boxes.”
What is ”interpretable”? Two possible perspectives:
Algorithmic transparency: decomposable, understandable, can easilyassign interpretations to parametersPost-hoc interpretation: Text, visualization, local explanation,explanation by example.
Linear models win on algorithmic transparency.
Neural networks win on post-hoc interpretation: rich features tovisualize, verbalize, cluster.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 9 / 35
![Page 25: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/25.jpg)
Interpretability: Fallacies
Fallacy 1. “Linear models are interpretable. Neural networks areblack boxes.”
What is ”interpretable”? Two possible perspectives:
Algorithmic transparency: decomposable, understandable, can easilyassign interpretations to parametersPost-hoc interpretation: Text, visualization, local explanation,explanation by example.
Linear models win on algorithmic transparency.
Neural networks win on post-hoc interpretation: rich features tovisualize, verbalize, cluster.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 9 / 35
![Page 26: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/26.jpg)
Interpretability: Fallacies
Fallacy 1. “Linear models are interpretable. Neural networks areblack boxes.”
What is ”interpretable”? Two possible perspectives:
Algorithmic transparency: decomposable, understandable, can easilyassign interpretations to parameters
Post-hoc interpretation: Text, visualization, local explanation,explanation by example.
Linear models win on algorithmic transparency.
Neural networks win on post-hoc interpretation: rich features tovisualize, verbalize, cluster.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 9 / 35
![Page 27: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/27.jpg)
Interpretability: Fallacies
Fallacy 1. “Linear models are interpretable. Neural networks areblack boxes.”
What is ”interpretable”? Two possible perspectives:
Algorithmic transparency: decomposable, understandable, can easilyassign interpretations to parametersPost-hoc interpretation: Text, visualization, local explanation,explanation by example.
Linear models win on algorithmic transparency.
Neural networks win on post-hoc interpretation: rich features tovisualize, verbalize, cluster.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 9 / 35
![Page 28: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/28.jpg)
Interpretability: Fallacies
Fallacy 1. “Linear models are interpretable. Neural networks areblack boxes.”
What is ”interpretable”? Two possible perspectives:
Algorithmic transparency: decomposable, understandable, can easilyassign interpretations to parametersPost-hoc interpretation: Text, visualization, local explanation,explanation by example.
Linear models win on algorithmic transparency.
Neural networks win on post-hoc interpretation: rich features tovisualize, verbalize, cluster.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 9 / 35
![Page 29: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/29.jpg)
Interpretability Definition 2: Post-hoc Explanation
Visualization. e.g. render distributed representations in 2D witht-SNE [9].
Local explanation. Popular: e.g., Saliency Maps [10], CAMs (classactivation mapping) [11], Grad-CAMs [12], attention [13, 14].
Figure: Grad-CAMs.
Explanation by example. Run k-NN on representations.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 10 / 35
![Page 30: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/30.jpg)
Interpretability Definition 2: Post-hoc Explanation
Visualization. e.g. render distributed representations in 2D witht-SNE [9].
Local explanation. Popular: e.g., Saliency Maps [10], CAMs (classactivation mapping) [11], Grad-CAMs [12], attention [13, 14].
Figure: Grad-CAMs.
Explanation by example. Run k-NN on representations.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 10 / 35
![Page 31: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/31.jpg)
Interpretability Definition 2: Post-hoc Explanation
Visualization. e.g. render distributed representations in 2D witht-SNE [9].
Local explanation. Popular: e.g., Saliency Maps [10], CAMs (classactivation mapping) [11], Grad-CAMs [12], attention [13, 14].
Figure: Grad-CAMs.
Explanation by example. Run k-NN on representations.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 10 / 35
![Page 32: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/32.jpg)
Interpretability Definition 2: Post-hoc Explanation
Visualization. e.g. render distributed representations in 2D witht-SNE [9].
Local explanation. Popular: e.g., Saliency Maps [10], CAMs (classactivation mapping) [11], Grad-CAMs [12], attention [13, 14].
Figure: Grad-CAMs.
Explanation by example. Run k-NN on representations.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 10 / 35
![Page 33: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/33.jpg)
Interpretability: Fallacies
Fallacy 2. “All AI applications need to be transparent.”
Figure: Is this a transparent algorithm? If not, why do you use it?
Transparency as a hard rule can exclude useful models that docomplex tasks better than us
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 11 / 35
![Page 34: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/34.jpg)
Interpretability: Fallacies
Fallacy 2. “All AI applications need to be transparent.”
Figure: Is this a transparent algorithm? If not, why do you use it?
Transparency as a hard rule can exclude useful models that docomplex tasks better than us
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 11 / 35
![Page 35: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/35.jpg)
Interpretability: Fallacies
Fallacy 2. “All AI applications need to be transparent.”
Figure: Is this a transparent algorithm? If not, why do you use it?
Transparency as a hard rule can exclude useful models that docomplex tasks better than us
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 11 / 35
![Page 36: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/36.jpg)
Interpretability: Fallacies
Fallacy 3. Always trust post-hoc explanation (e.g., CAMs).
Post-hoc interpretations can be optimized to mislead.
E.g., in college admissions, post-hoc explanations of leadership andoriginality disguise racial, gender discrimination [15].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 12 / 35
![Page 37: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/37.jpg)
Interpretability: Fallacies
Fallacy 3. Always trust post-hoc explanation (e.g., CAMs).
Post-hoc interpretations can be optimized to mislead.
E.g., in college admissions, post-hoc explanations of leadership andoriginality disguise racial, gender discrimination [15].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 12 / 35
![Page 38: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/38.jpg)
Interpretability: Fallacies
Fallacy 3. Always trust post-hoc explanation (e.g., CAMs).
Post-hoc interpretations can be optimized to mislead.
E.g., in college admissions, post-hoc explanations of leadership andoriginality disguise racial, gender discrimination [15].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 12 / 35
![Page 39: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/39.jpg)
Interpretability: Summary
Never discuss “interpretability” without clarifying the definition.
Beware of interpretability fallacies.
Find your domain-specific definition of interpretability, then use thetools available.
Align evaluation metrics with what is qualitatively important
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 13 / 35
![Page 40: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/40.jpg)
Interpretability: Summary
Never discuss “interpretability” without clarifying the definition.
Beware of interpretability fallacies.
Find your domain-specific definition of interpretability, then use thetools available.
Align evaluation metrics with what is qualitatively important
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 13 / 35
![Page 41: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/41.jpg)
Interpretability: Summary
Never discuss “interpretability” without clarifying the definition.
Beware of interpretability fallacies.
Find your domain-specific definition of interpretability, then use thetools available.
Align evaluation metrics with what is qualitatively important
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 13 / 35
![Page 42: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/42.jpg)
Interpretability: Summary
Never discuss “interpretability” without clarifying the definition.
Beware of interpretability fallacies.
Find your domain-specific definition of interpretability, then use thetools available.
Align evaluation metrics with what is qualitatively important
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 13 / 35
![Page 43: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/43.jpg)
Expense: Data and compute
Switching gears: ML can be expensive.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 14 / 35
![Page 44: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/44.jpg)
Expense: Data
Costly data collection and computation (in time and money).
Solution 1: Unsupervised [16, 17] and semi-supervised approaches[18].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 15 / 35
![Page 45: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/45.jpg)
Expense: Data
Costly data collection and computation (in time and money).
Solution 1: Unsupervised [16, 17] and semi-supervised approaches[18].
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 15 / 35
![Page 46: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/46.jpg)
Expense: Data
Transfer learning [17, 19]. Pretrain on related tasks.
Use public datasets, e.g., ImageNet.Download model parameters from internet.Recent work from Stanford researchers: Taskonomy [20].
Figure: Taskonomy: “taxonomy of tasks” to guide transfer learning - modelingthe structure of space of visual tasks
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 16 / 35
![Page 47: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/47.jpg)
Expense: Data
Transfer learning [17, 19]. Pretrain on related tasks.Use public datasets, e.g., ImageNet.
Download model parameters from internet.Recent work from Stanford researchers: Taskonomy [20].
Figure: Taskonomy: “taxonomy of tasks” to guide transfer learning - modelingthe structure of space of visual tasks
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 16 / 35
![Page 48: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/48.jpg)
Expense: Data
Transfer learning [17, 19]. Pretrain on related tasks.Use public datasets, e.g., ImageNet.Download model parameters from internet.
Recent work from Stanford researchers: Taskonomy [20].
Figure: Taskonomy: “taxonomy of tasks” to guide transfer learning - modelingthe structure of space of visual tasks
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 16 / 35
![Page 49: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/49.jpg)
Expense: Data
Transfer learning [17, 19]. Pretrain on related tasks.Use public datasets, e.g., ImageNet.Download model parameters from internet.Recent work from Stanford researchers: Taskonomy [20].
Figure: Taskonomy: “taxonomy of tasks” to guide transfer learning - modelingthe structure of space of visual tasks
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 16 / 35
![Page 50: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/50.jpg)
Expense: Compute
Since Deep Learning, compute use has been increasing faster than Moore’sLaw! Popular media: Training a single AI model can emit as much carbonas five cars in their lifetimes
Figure: From OpenAI
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 17 / 35
![Page 51: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/51.jpg)
Expense: Compute
Compression [21].
Quantization [22].
Specialized hardware [23, 24]. GPUs are inefficient. More efficiencywith FPGA, TPU.
Figure: Deep compression: Pruning connections, quantizing weights, and Huffmancoding (shorter codes for higher frequencies of occurrence) (50× gains.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 18 / 35
![Page 52: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/52.jpg)
Expense: Compute
Compression [21].
Quantization [22].
Specialized hardware [23, 24]. GPUs are inefficient. More efficiencywith FPGA, TPU.
Figure: Deep compression: Pruning connections, quantizing weights, and Huffmancoding (shorter codes for higher frequencies of occurrence) (50× gains.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 18 / 35
![Page 53: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/53.jpg)
Expense: Compute
Compression [21].
Quantization [22].
Specialized hardware [23, 24]. GPUs are inefficient. More efficiencywith FPGA, TPU.
Figure: Deep compression: Pruning connections, quantizing weights, and Huffmancoding (shorter codes for higher frequencies of occurrence) (50× gains.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 18 / 35
![Page 54: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/54.jpg)
Summary: Expense
Data: Transfer learning, public datasets, unsupervised pretraining.Newer techniques coming out frequently.
Compute: Compression, quantization, specialized hardware.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 19 / 35
![Page 55: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/55.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 56: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/56.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 57: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/57.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].
Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 58: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/58.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.
Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 59: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/59.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 60: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/60.jpg)
Community weaknesses
Cycle of hype and winter [25].
Lack of rigor and worries of troubling scholarship trends [26, 27].
Many incorrect theories invented to explain observations, rather thanderived from theoretical foundations [28, 29].Suggestion of [28]: Spend more time doing experiments to find rootcause for unexpected results, rather than chasing performance.Barriers to entry (funding and data).
Side-effects of industry-driven research?
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 20 / 35
![Page 61: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/61.jpg)
Ethical Concerns
Last gear switch: some things to be mindful of wrt ML.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 21 / 35
![Page 62: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/62.jpg)
Ethical Concerns
ML captures social biases in dataset
Like any technology, ML can be used in ways whose legality / ethicsare questionable
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 22 / 35
![Page 63: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/63.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 64: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/64.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 65: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/65.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 66: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/66.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 67: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/67.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 68: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/68.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)
Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 69: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/69.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned men
AI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 70: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/70.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 71: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/71.jpg)
ML captures social biases in dataset
NYT (11/11/19): ”We Teach A.I. Systems Everything, Including OurBiases”
”BERT is more likely to associate the word “programmer” with menthan with women.”
”If a tweet or headline contained the word “Trump,” the tool almostalways judged it to be negative, no matter how positive thesentiment.”
NYT (06/17/19) ”Exposing the Bias Embedded in Tech”
Imbalance in training data leads to negative societal consequences
Xbox Kinect (2010) worked less well for women and children (trainedon 18-35 year old men)Facial recognition more accurate with lighter-skinned menAI resume readers penalized occurrences of ”women” and ”women’scolleges”
Pro Publica (2016) ”Machine Bias” - race and AI risk assessments / bailcalculations
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 23 / 35
![Page 72: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/72.jpg)
Questionable Use of AI
CNN (01/2019): ”When seeing is no longer believing - Inside thePentagon’s race against deepfake videos”
Eroding trustworthiness of video evidence
VICE (06/27/19): ”Creator of DeepNude, App That Undresses Photos ofWomen, Takes It Offline”
Legality and legal rights over deepfakes
NYT (04/14/19): ”One Month, 500,000 Face Scans: How China Is UsingA.I. to Profile a Minority”
Billion-dollar companies with government contracts for masssurveillance
Legal frameworks for holding AI users accountable are needed
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 24 / 35
![Page 73: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/73.jpg)
Questionable Use of AI
CNN (01/2019): ”When seeing is no longer believing - Inside thePentagon’s race against deepfake videos”
Eroding trustworthiness of video evidence
VICE (06/27/19): ”Creator of DeepNude, App That Undresses Photos ofWomen, Takes It Offline”
Legality and legal rights over deepfakes
NYT (04/14/19): ”One Month, 500,000 Face Scans: How China Is UsingA.I. to Profile a Minority”
Billion-dollar companies with government contracts for masssurveillance
Legal frameworks for holding AI users accountable are needed
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 24 / 35
![Page 74: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/74.jpg)
Questionable Use of AI
CNN (01/2019): ”When seeing is no longer believing - Inside thePentagon’s race against deepfake videos”
Eroding trustworthiness of video evidence
VICE (06/27/19): ”Creator of DeepNude, App That Undresses Photos ofWomen, Takes It Offline”
Legality and legal rights over deepfakes
NYT (04/14/19): ”One Month, 500,000 Face Scans: How China Is UsingA.I. to Profile a Minority”
Billion-dollar companies with government contracts for masssurveillance
Legal frameworks for holding AI users accountable are needed
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 24 / 35
![Page 75: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/75.jpg)
Questionable Use of AI
CNN (01/2019): ”When seeing is no longer believing - Inside thePentagon’s race against deepfake videos”
Eroding trustworthiness of video evidence
VICE (06/27/19): ”Creator of DeepNude, App That Undresses Photos ofWomen, Takes It Offline”
Legality and legal rights over deepfakes
NYT (04/14/19): ”One Month, 500,000 Face Scans: How China Is UsingA.I. to Profile a Minority”
Billion-dollar companies with government contracts for masssurveillance
Legal frameworks for holding AI users accountable are needed
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 24 / 35
![Page 76: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/76.jpg)
Conclusion
Technical and societal critiques of AI: we’ve only scratched the surface.
1 Adversarial examples
2 Interpretability
3 Expense: Data and compute
4 Community weaknesses
5 Ethical Concerns
ML is a dynamic field with wide-reaching societal impact. Take yourcritics and stakeholders seriously!
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 25 / 35
![Page 77: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/77.jpg)
References I
[1] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna,Dumitru Erhan, Ian Goodfellow, and Rob Fergus.Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013.
[2] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.Explaining and harnessing adversarial examples.arXiv preprint arXiv:1412.6572v3, 2015.
[3] Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon.Generative adversarial examples.arXiv preprint arXiv:1805.07894, 2018.
[4] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 26 / 35
![Page 78: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/78.jpg)
References II
[5] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, andAnanthram Swami.Distillation as a defense to adversarial perturbations against deepneural networks.In 2016 IEEE Symposium on Security and Privacy (SP), pages582–597. IEEE, 2016.
[6] Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, andNate Kushman.Pixeldefend: Leveraging generative models to understand and defendagainst adversarial examples.arXiv preprint arXiv:1710.10766, 2017.
[7] Anish Athalye and Ilya Sutskever.Synthesizing robust adversarial examples.arXiv preprint arXiv:1707.07397, 2017.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 27 / 35
![Page 79: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/79.jpg)
References III
[8] Zachary C Lipton.The mythos of model interpretability.arXiv preprint arXiv:1606.03490, 2016.
[9] Laurens van der Maaten and Geoffrey Hinton.Visualizing data using t-sne.Journal of machine learning research, 9(Nov):2579–2605, 2008.
[10] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.Deep inside convolutional networks: Visualising image classificationmodels and saliency maps.arXiv preprint arXiv:1312.6034, 2013.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 28 / 35
![Page 80: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/80.jpg)
References IV
[11] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, andAntonio Torralba.Learning deep features for discriminative localization.In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, pages 2921–2929, 2016.
[12] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das,Ramakrishna Vedantam, Devi Parikh, Dhruv Batra, et al.Grad-cam: Visual explanations from deep networks via gradient-basedlocalization.In ICCV, pages 618–626, 2017.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 29 / 35
![Page 81: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/81.jpg)
References V
[13] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville,Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio.Show, attend and tell: Neural image caption generation with visualattention.In International conference on machine learning, pages 2048–2057,2015.
[14] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, LlionJones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.Attention is all you need.In Advances in Neural Information Processing Systems, pages5998–6008, 2017.
[15] Opinion — is harvard unfair to asian-americans? - the new yorktimes.https://www.nytimes.com/2014/11/25/opinion/
is-harvard-unfair-to-asian-americans.html?_r=0, 2014.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 30 / 35
![Page 82: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/82.jpg)
References VI
[16] Honglak Lee, Peter Pham, Yan Largman, and Andrew Y Ng.Unsupervised feature learning for audio classification usingconvolutional deep belief networks.In Advances in neural information processing systems, pages1096–1104, 2009.
[17] Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-AntoineManzagol, Pascal Vincent, and Samy Bengio.Why does unsupervised pre-training help deep learning?Journal of Machine Learning Research, 11(Feb):625–660, 2010.
[18] Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, andMax Welling.Semi-supervised learning with deep generative models.In Advances in Neural Information Processing Systems, pages3581–3589, 2014.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 31 / 35
![Page 83: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/83.jpg)
References VII
[19] Kaiming He, Ross Girshick, and Piotr Dollar.Rethinking imagenet pretraining.arXiv preprint arXiv:1811.08883, 2018.
[20] Amir R Zamir, Alexander Sax, William Shen, Leonidas Guibas,Jitendra Malik, and Silvio Savarese.Taskonomy: Disentangling task transfer learning.In Proceedings of the IEEE Conference on Computer Vision andPattern Recognition, pages 3712–3722, 2018.
[21] Song Han, Huizi Mao, and William J Dally.Deep compression: Compressing deep neural networks with pruning,trained quantization and huffman coding.arXiv preprint arXiv:1510.00149, 2015.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 32 / 35
![Page 84: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/84.jpg)
References VIII
[22] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, andYoshua Bengio.Quantized neural networks: Training neural networks with lowprecision weights and activations.The Journal of Machine Learning Research, 18(1):6869–6898, 2017.
[23] Stephen D Brown, Robert J Francis, Jonathan Rose, and Zvonko GVranesic.Field-programmable gate arrays, volume 180.Springer Science & Business Media, 2012.
[24] Norm Jouppi.Google supercharges machine learning tasks with tpu custom chip.Google Blog, May, 18, 2016.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 33 / 35
![Page 85: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/85.jpg)
References IX
[25] Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio.Deep learning, volume 1.MIT press Cambridge, 2016.
[26] Zachary C Lipton and Jacob Steinhardt.Troubling trends in machine learning scholarship.arXiv preprint arXiv:1807.03341, 2018.
[27] Theories of deep learning (stats 385).https://stats385.github.io/readings, 2017.
[28] Ali Rahimi.Ai is the new alchemy (nips 2017 talk).https://www.youtube.com/watch?v=Qi1Yry33TQE, December2017.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 34 / 35
![Page 86: Technical and Societal Critiques of MLcs229.stanford.edu/materials/critiques-ml-aut19.pdf · Technical and Societal Critiques of ML CS 229, Fall 2019 slides by Chris Chute, Taide](https://reader036.vdocument.in/reader036/viewer/2022071020/5fd445802954810cf7309b00/html5/thumbnails/86.jpg)
References X
[29] Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and AleksanderMadry.How does batch normalization help optimization?(no, it is not aboutinternal covariate shift).arXiv preprint arXiv:1805.11604, 2018.
CS 229, Fall 2019 (Stanford) ML Critiques November 15, 2019 35 / 35