dsai.se cse, chalmers · •machine learning predictions are more accurate for some •ai system...

AlgorithmicFairness&MachineLearningFredrikD.Johansson

DSAI.se CSE,Chalmers

2

Artificialintelligence(AI)isalreadypartofsociety

Autonomoustransportation

Recommendationsystems

“Smart”homes

Clinicaldecisionsupport

3

Decisionmaking

Illustration:De-Arteaga,2019,http://demo.clab.cs.cmu.edu/ethical_nlp/slides/BiasInBios.pdf

4

MLindecisionmaking


Prediction offuture crime

Prediction ofjob success

Prediction ofrisk of death

5

Supervisedmachinelearning(ML)

• Systemsthatlearn topredictlabel 𝒀 forinput 𝑿

• Example:Recognition

Whatisthis? 𝑿 = , 𝒀 = ”dog”

• Data:Labeled images

𝑿 = , 𝒀 = ”cat”

…

6

SupervisedlearningII

1Dengetal.,CVPR,2009

1.Observation 2.Prediction

“Cat”Supervisedlearningloop

“Dog”“Cat”

3.Supervision4.Learning

Categorizeimagesinto1000classes1

0,0%5,0%10,0%15,0%20,0%25,0%30,0%

Errorinpredictingcorrectlabel

7

Empiricalriskminimization

• Findthemodel𝜽 withtheleastobservedpredictionmistakes

Learning

minimize`

𝔼b 𝑌d̀ ≠ 𝑌

Predictedlabel

Frequencyofobserved mistakes

Truelabel

Data

8

MLindecisionmaking


Prediction offuture crime

Whathappenswhenthispredictionisbiased?

Whatifdecisionsmadebasedonitarediscriminatory?

9

Biaseddata&biasedalgorithms

• Machinelearningpredictionsaremoreaccurateforsome

• AIsystemforbaildecisions ismorelenientforwhitepeople,morestrictforblackpeople1

• MLmaypreserve humanbiasesorcreatenewones—orboth!

1Angwinetal.ProPublica.2016

MachineBias1

10

Biaseddata&biasedalgorithms

• Machinelearningpredictionsaremoreaccurateforsome

• AIsystemforbaildecisions ismorelenientforwhitepeople,morestrictforblackpeople1

• MLmaypreservehumanbiasesorcreatenewones—orboth!


MachineBias1

Howcanweavoidthis?

11

Outline

1. Definitionsoffairnessandbias

2. Reducingbias

3. Limitationsofgroupfairness

4. Wrappingup

12

Shout-outtoMoritzHardt’s andSolonBarocas’

NeurIPS ‘17TutorialonFairnessinMachineLearning

Lookitupformoredetails&examples!

https://mrtz.org/nips17

13

Algorithmicfairness

• Asystemmaybedeemedunfairifitdiscriminatesindividualsbasedoninformationthatisirrelevant tothesystem’spurpose

• Thedetailsareinherentlydomain-specific

• Algorithmicfairnessattemptstoformalize thismathematically

14

Exampleofunfairness

• Scenario: Onaverage,theriskofblackcriminalscommittingcrimesafterbeingreleasedisoverestimated moreoftenthantheriskforwhitecriminals.

• Thisisaconcernstatedonthegrouplevel• Wewillseeotherexampleslater!

• Thestatementistiedtotheattributes ofblack/white

15

Protectedattributes

• ItisillegalinSwedentodiscriminateonthebasisofsex,gender,ethnicity,religion,disability,sexualpreferenceorage1

• Theseareexamplesofso-calledprotectedattributes

• Wewillfirstdiscussfairnessw.r.t.these

1 https://www.do.se/lag-och-ratt/diskrimineringslagen/

16

Direct&indirectdiscrimination

• Directdiscrimination(disparatetreatment):Individualsare explicitly treateddifferentlyonthebasisofaprotectedattribute

• Indirectdiscrimination(disparateimpact):Individualswithcertainprotectedattributesaredisadvantagedasaresultofseeminglyneutral policies

17

Actingonpredictions

• Forsimplicity,weassumethatthedecision-makerisonlyinterestedinthetargetofprediction1

1I.e.,if𝑌 wasknown,wewouldbaseourdecisiononthat

Prediction of future crime

Protected attribute A

Future crime(unobserved)

𝑌d(𝑋, 𝐴)

𝑌

X

18

Howtoformalizefairness?

• Attempt1:IndependenceToavoiddirect discrimination,enforceindependencebetweenprediction𝑌d andtheprotectedattribute𝐴

• orequivalently,Pr 𝑌d = 1 ∣ 𝐴 = 0 = Pr 𝑌d = 1 ∣ 𝐴 = 1

𝑌d ⊥ 𝐴 𝑌d andA statisticallyindependent

19

Independencecriterion

• Independenceisaverycrude metric• Alsocalledstatisticalparity,demographicparity

• Doesnottakeintoaccountthecontext𝑋 (e.g.,criminalrecord)ortheprobabilityoftheoutcome𝑌 (e.g.,recidivism)• Allowsaccuracyinonegroup,randomnessintheother

• Outcome𝑌 couldbecorrelatedwith 𝐴• Then,theperfectprediction𝑌d = 𝑌 doesnotsatisfyindependence

𝑌d ⊥ 𝐴

20


• Attempt2:SeparationAttemptstoeliminaterelianceonprotectedattributegivenperfectinformationoftheoutcome

• orequivalently,Pr 𝑌d ∣ 𝐴 = 0, 𝑌 = Pr 𝑌d ∣ 𝐴 = 1, 𝑌

𝑌d ⊥ 𝐴 ∣ 𝑌

21

Separationcriterion

• Fixessomeshort-comingswithindependence:• Allowsperfectprediction:𝑌d = 𝑌

• Capturesequalityinopportunity,

𝑌d ⊥ 𝐴 ∣ 𝑌

Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 = 1 = Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌 = 1

Giventhatindividualswithdifferentprotectedattributes arebothgoingtosucceed,theyaregiventhesameopportunity

22


• Attempt3:SufficiencyAttemptstoobtainaprediction𝑌dtowhichadding𝐴 givesnoextrainformationabouttheoutcome𝑌

• orequivalently,Pr 𝑌 ∣ 𝐴 = 0, 𝑌d = Pr 𝑌 ∣ 𝐴 = 1, 𝑌d

𝑌 ⊥ 𝐴 ∣ 𝑌d

23

Sufficiencycriterion

• Sufficiencyholdsforgroup-calibrated models

• Let𝑅 denotethescoreusedtodetermine𝑌d

• Forexample𝑌d = 1 𝑅 > 𝑡 forsomethresholdt

• Then,𝑅 iscalibratedforgroup𝑎 if

𝑌 ⊥ 𝐴 ∣ 𝑌d

Pr 𝑌 = 1 ∣ 𝑅 = 𝑟, 𝐴 = 𝑎 = 𝑟

Theprobabilityoftheoutcomewhenweassignscore𝑟isequalto𝑟

24

Statisticalfairnesssummary

Independence Separation Sufficiency

𝑌d ⊥ 𝐴 𝑌d ⊥ 𝐴 ∣ 𝑌 𝑌 ⊥ 𝐴 ∣ 𝑌d

• Thesearepotentiallyalldesiredpropertiesofpredictivesystems

Statisticalparity E.g.,equalopportunity Calibration

25

Outline


2. Reducingbias


4. Wrappingup

26

Wheredoesbiasenter?


Dataencodespre-existinghumanbiases

Machinesmayamplifyorcreatebias

Humansactbiased onpredictions

27

Wheredoesbiasenter?


Let’sfocusonthisone

28

Biasinlearning

• Assumethatourdataisunbiased,i.e.,thatwelearntopredictbasedonthethingsweactuallycareabout• E.g.,recidivismvspreviousbaildecisions

• Now,saywewanttoensureseparation(e.g.,equalityinopportunity)

• Whycan’twejustdosupervisedlearningasnormal?

Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 = Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌

29

Biasinlearning

• Underseparation,weareconcernedwithunfairnessinerrors

Pr 𝑌d = 0 ∣ 𝐴 = 0, 𝑌 = 1 ≠ Pr 𝑌d = 0 ∣ 𝐴 = 1, 𝑌 = 1Differenceinfalsenegativerates (underestimatingrisk)

30

Biasinlearning

• Underseparation,weareconcernedwithunfairnessinerrors

• If𝑌d = 𝑌,separationissatisfied(unlikely)

Pr 𝑌d = 0 ∣ 𝐴 = 0, 𝑌 = 1 ≠ Pr 𝑌d = 0 ∣ 𝐴 = 1, 𝑌 = 1

Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 = 0 ≠ Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌 = 0Differenceinfalsepositiverates (overestimatingrisk)

Differenceinfalsenegativerates (underestimatingrisk)

31

Supervisedlearningrecap

• Recall: Supervisedlearningattemptstominimizeexpectederror

• Whatiftheerrorisdifferentfordifferentprotectedgroups?

• Theobjectiveabovedoesnothingtopromotefairness• Neitherdoesstratifyingitbygroupandminimizingseparately

minimize`

𝔼b 𝑌d̀ ≠ 𝑌

32

Decomposingpredictionerror

• Predictionerrors,falsepositiveratesandmeansquarederrorsmayallbedecomposedintermsofbias,varianceandnoise1

• Decomposingerrormayguidereductionofbias!2

1Domingos,2000,2Chen,J.,Sontag,NeurIPS,2018

Predictionerror = Bias + c�Variance + c�Noise

Errordueto…poormodel … smallsample … inadequatecovariates

33

Decomposingunfairness

• Violations ofourfairnesscriterion

• maybedecomposed asdifferencesinbias,variance,noise1

1Chen,J.,Sontag,NeurIPS,2018

Γ = Bias� − Bias� + Variance� − Variance�+ Noise� − Noise�

Γ = Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 − Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌

34

Example:Differentvariance

• Muchfewersamplesforonegroup

SubjectfromgroupASubjectfromgroupB

Riskofrecidivism,𝑌

Context,𝑋

35


• MuchfewersamplesforonegroupTrueriskgroupB

TrueriskgroupA

SubjectfromgroupASubjectfromgroupB

36


• Muchfewersamplesforonegroup

ModelforB

ModelforA

37

Example:Predictingfutureincome1

• ExaminingimpactofvarianceonunfairnessintheAdultdatasetforpredictionhigh/lowincome

• Collectingmoresamplesreducesthedifferenceinfalsepositiveratesandfalsenegativerates

1Chen,J.,Sontag,NeurIPS,2018

Unfairn.FalsePositivesUnfairn.FalseNegatives

38

Example:Differentbias

• Modelbettersuitedtoonegroup

ModelforA

ModelforB

39

Example:DifferentNoise

• Context𝑋 morepredictiveforonegroup

Context,𝑋

40

Combattingsourcesofunfairness

• HigherbiasforgroupA?

• HighervarianceforgroupA?

• HighernoiseforgroupA?

TailorthemodeltoA

Collect moresamplesfromgroupA

MeasuremorevariablesrelevanttogroupA

41

Combatting sourcesofunfairness

• HigherbiasforgroupA?

• HighervarianceforgroupA?

• HighernoiseforgroupA?

• WhatifI’vedoneallIcan?

TailorthemodeltoA

CollectmoresamplesfromgroupA

MeasuremorevariablesrelevanttogroupA

42

Enforcingfairnesscriteria1

• Alltheremediesonthepreviousslideattempttoreduceerrorforthegroupwithhighererror

• Ifthisisinfeasible,andwearewillingtosacrificesomeaccuracy,wecanconstrain orpost-process predictionstosatisfycriteria

• Example: LearnonlymodelssuchthatPr 𝑌d = 1 ∣ 𝐴 = 0 = Pr 𝑌d = 1 ∣ 𝐴 = 1 (independence)

1Hardt,Price,Srebro,NeurIPS,2016

43

Outline


2. Reducingbias


4. Wrappingup

44

Statisticalfairnesssummary


𝑌d ⊥ 𝐴 𝑌d ⊥ 𝐴 ∣ 𝑌 𝑌 ⊥ 𝐴 ∣ 𝑌d

• Theseareoftenalldesiredpropertiesofpredictivesystems

45

Statisticalfairnessincompatibility


𝑌d ⊥ 𝐴 𝑌d ⊥ 𝐴 ∣ 𝑌 𝑌 ⊥ 𝐴 ∣ 𝑌d

• Problem: Innon-trivialcaseswhere𝑌 ⊥ 𝐴 or𝑌d ⊥ 𝑌,anytwoofthesethreecriteriaaremutuallyexclusive—wehavetochoose!

46

Machinebias—asecondlook


1. Theriskscore,COMPAS,thatwasusedtopredictrecidivismwasclaimedtobecalibrated

• Theirriskscoresatisfied

MachineBias1

Pr 𝑌 = 1 ∣ 𝑅 = 𝑟, 𝐴 = 𝑎 = 𝑟

Approximatelysatisfiedsufficiency

47



2. ThepublicationProPublicashowedthatfalsenegativerates andfalsepositiverateswere differedacrossraces

Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 = 0≠ Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌 = 0



Pr 𝑌 = 1 ∣ 𝑅 = 𝑟, 𝐴 = 𝑎 = 𝑟

Approximatelysatisfiedsufficiency Didnotsatisfyseparation

48



2. ThepublicationProPublicashowedthatfalsenegativerates andfalsepositiverateswere differedacrossraces

Pr 𝑌d = 1 ∣ 𝐴 = 0, 𝑌 = 0≠ Pr 𝑌d = 1 ∣ 𝐴 = 1, 𝑌 = 0



Pr 𝑌 = 1 ∣ 𝑅 = 𝑟, 𝐴 = 𝑎 = 𝑟

Whichdowecaremoreabout?

Approximatelysatisfiedsufficiency Didnotsatisfyseparation

49


• Neithercalibrationnorseparationruleoutunfairpractices

• Forexample,calibrationisinsensitive todifferencesintruerisk

Pr 𝑌� 0.1 0.2 0.4 0.6 0.8 0.9

Average

=0.5

𝑌�� 0.5 0.5 0.5 0.5 0.5 0.5 Calibrated!Overestimaterisk Underestimaterisk

50


• Neithercalibrationnorseparationruleoutunfairpractices

• Forexample,calibrationisinsensitive todifferencesintruerisk

Pr 𝑌� 0.1 0.2 0.4 0.6 0.8 0.9

Average

=0.5

𝑌�� 0.5 0.5 0.5 0.5 0.5 0.5 Calibrated!

Unfairevenwithinthegroup!

Overestimaterisk Underestimaterisk

51

1. Comparesoutcomesforcomparablesubjects

2. Notionof“comparable”shouldbetask-specific

3. Hardtomakepractical

Individualfairness

Group&individualfairness

Statistical/groupfairness

1. Comparesstatisticsofprotectedgroups

2. Differentstatisticstelldifferentstories

3. Definitionanduseofgroupsisoftensensitive

52

Counterfactualfairness

• Individualfairnessisintimatelytiedtocausality

IfIwasawoman,wouldIhavebeenlesslikelytogethired?

• Thisisacounterfactualquestion1.Largeliteratureforthese,someofwhichhasbeenadaptedtofairness2,3

• Difficulttospecifywhat“ifIwasawoman”means• Canuseproxies—e.g.,putafemalenameontheCV

1Pearl,Causality,2000,2Kusneretal.,NeurIPS,2017,3Nabi&Shpitser,AAAI,2018

53

Biaseddata

• Sofar,we’veassumedthat𝑌,theoutcomeitself,isfair

• Measurementof𝑌 mightitselfbebiased.Forexample,if• 𝑌 representspasthiringdecisions• or𝑌 isanaggregateoffeaturesbettersuitedforonegroup• or𝑌 isoutdated(e.g.,anoutcomeunderanoldpolicy)• …

54

Outline


2. Reducingbias


4. Wrappingup

55

Wrappingup

Thereisnoone-size-fits-allfairnessandwecan’tsatisfyallalternatives

• Wefundamentally havetochoosedependingondomain

56

Wrappingup

Statisticaldifferencesdon’talwaysimplydiscrimination(correlation≠ causation)

• Differentfairnessinterpretationsforsamestatistics

57

Wrappingup

TheseissuesarenotspecifictoAI!

• Sameproblemsforhumans,rule-basedsystems,andAIalike…butweneedtomonitorallofthese

58

FredrikD.Johansson

[email protected]

DSAI.se

IreneChen DavidSontag

Workedonfairnesswith

dsai.se cse, chalmers · •machine learning predictions are more accurate for some •ai system...

Documents