ArtificiaI Intelligence, Bias and Discrimination A European perspective

osborneclarke.com

Agenda• Setting the context

― What is bias?

― Human (cognitive) bias classes

• Bias and machine learning

― Types of automated bias

― Examples

• The Equality Act 2010

― Protected Characteristics

― Forms of discrimination

― Application to AI

― Remedies

• The EU GDPR

― Automated Decision Making (again)

― Special Category Data

• Q&A

osborneclarke.com

Setting the context: What is bias?

• Lack of fairness ?

• Lack of objectivity ?

"a disproportionate weight in favour of or against an idea or thing,

usually in a way that is closed-minded, prejudicial, or unfair. Biases

can be innate or learned. People may develop biases for or against

an individual, a group, or a belief. In science and engineering, a bias

is a systematic error."

- Wikipedia

osborneclarke.com

Setting the context: What is bias?

• Is the absence of bias (fairness) the introduction of objectivity or reality?

osborneclarke.com

Setting the context: What is bias?

• Is the absence of bias (fairness) the introduction of objectivity or reality?

No – it is really the application of different subjectivity

conditioned by societal (human) norms

osborneclarke.com

Setting the context: Cognitive (human) bias

• There are many types:

Anchoring:

Over reliance on the first

piece of information you

hear

Availability heuristic:

Overestimating the

importance of info

available to you (personal

experience)

Bandwagon:

Probability of holding a

belief increases (and gains

credibility) based on the

number of people holding

it

Blind spot bias:

Failing to recognise your

own bias is itself a bias

Choice supportive bias:

When you make a choice

you tend to feel positive

about it – even if it has

flaws

Clustering:

A tendency to see patterns

in random events (ie

gambling fallacies)

Confirmation bias:

Only listening to

information that confirms

our preconceptions

Ostrich effect:

Deciding to ignore

dangerous or negative

information

Outcome bias:

Judging a decision based

on the outcome rather

than how the decision was

made

Placebo effect:

Believing something to

have an effect causes it to

have that effect

Recency:

Tendency to weigh latest

information more heavily

than older data.

Salience:

Tendency to focus on most

easily recognisable

features of a concept.

Stereotyping:

Expecting a group or

person to have certain

qualities without any real

information about the

individual

Survivorship bias:

Focus only on surviving

examples, causing us to

misjudge a situation

Zero Risk bias:

A love of certainty –

eliminating risk means that

there is no chance of harm

osborneclarke.com

Bias and Machine Learning

• A complex question – particularly with AI and Machine Learning systems:

He is lazy O tembel

O tembel She's lazy

She is smart O akilli

O akilli He's smart

He is a nurse Hän on

sairaanhoitaja

Hän on

sairaanhoitaja

She's a nurse

She is an

engineer

Han on insinööri

Han on insinööri He is an engineer

Google Translate

English → Turkish → English English → Finnish → English

osborneclarke.com

Bias and Machine Learning

• Types of AI bias:

Measurement Bias

Weighting Bias

Selection Bias

Occurs when the

Data used to power the

Algorithmic model are not

Fully representative of

The actual demographic context

Occurs when the

Data collection vector (process)

causes the data to be skewed in

a particular manner

Occurs when the

Algorithmic model applies differing

"weights" to different datasets,

skewing an outcome arbitrarily or

inaccurately

osborneclarke.com

Bias and Machine Learning

• Examples:

State of Wisconsin v. Loomis - COMPAS Automated Recruitment tool

Access control

osborneclarke.com

Equality Act 2010: Protected Characteristics

Sex

Religion or

belief

Marriage &

civil

partnership

Race

Sexual

orientation

Pregnancy &

maternity

Age

Gender re-

assignmentDisability

Protected

characteristics

• Protected Characteristics:

osborneclarke.com

Equality Act 2010: Forms of Discrimination

Direct Discrimination

Indirect Discrimination

Victimisation

Harassment

Because of a "protected characteristic", a person (A) treats

another (B) less favourably than A treats or would treat others. A

comparator is required.

A applies to B an apparently neutral provision, criterion or

practice that A would apply equally to others, but which puts or

would put those who share B's protected characteristic at a

particular disadvantage.

A treats another B unfavourably because A knows or suspects B

has done or intends to do a "protected act" (ie reporting conduct

under the EA)

A engages in unwanted conduct related to a protected

characteristic (or of a sexual nature) that has the purpose or

effect of violating B's dignity or creating an intimidating, hostile,

degrading, humiliating or offensive environment for B

osborneclarke.com

Equality Act 2010: Application to AI and Machine Learning

• Normally applied in employment context

• Also applies to "Service Providers" (s29)

"…a person…concerned with the provision of a service to the public or a

section of the public (for payment or not)…"

• Not a defence to claim ignorance of technology (or that did not know or intend

to discriminate). Service providers will normally be fixed with the

discriminatory consequences of the technology, even if they do not know

how it works.

osborneclarke.com

Equality Act 2010: Application to AI and Machine Learning

• s136: Burden of Proof:

"If there are facts from which the court could decide, in the absence of any

other explanation, that a person (A) contravened the provision concerned, the

court must hold that the contravention occurred."

• The EA does not allow for a right to understand decisions made by AI (see

Future of Work Commission Report, 2017)

osborneclarke.com

Direct Discrimination

Equality Act 2010: Application to AI and Machine Learning

Etsy contacting users on Valentines day and its algorithm

assuming that female users were in relationships with men…the

problem is that Maggie is a lesbian…

osborneclarke.com

Indirect Discrimination

Equality Act 2010: Application to AI and Machine Learning

Facebook's name verification algorithm removes "Shane

Creepingbear" from its platform as it does not conform to its

westernised name model. Shane is in fact a native of the Kiowa

tribe in Oklahoma…

osborneclarke.com

HarassmentGoogle photo descriptors label a black user and her friend as

"gorillas"…

Equality Act 2010: Application to AI and Machine Learning

osborneclarke.com

A tribunal may award

compensation in a

successful discrimination

claim for both financial

losses and injury to feelings.

In addition it may make:

- A declaration

- A recommendation

Compensation can be

awarded against both the

company and the individual

perpetrator.

Equality Act 2010: Consequences of successful claims

osborneclarke.com

Lower Band

£900 to £8,600

Middle Band

£8,600 to £25,700

Higher Band

£25,700 to £42,900

18

Equality Act 2010: Injury to feelings "Vento" damages scale

osborneclarke.com

GDPR: Article 22

"The data subject shall have the right not to be subject to a decision based solely on automated

processing, including profiling, which produces legal effects concerning him or her or similarly

significantly affects him or her. … Paragraph 1 shall not apply if the decision…is based on the data

subject's explicit consent"

Automated decision making, Profiling

osborneclarke.com

Article 9

• Additional rules and rigour apply to Special Category data

• These are the protected characteristics of an individual (ie race, ethnic origin,

religion, genetic data, biometric data used for identification, health and sexual

characteristics)

• Special category data could also be subject to ADM

• Article 9(2)(e) exception for making the information “manifestly public” does not

apply when ADM used

• As with ADM, explicit consent normally required as a lawful basis

Special Category Data

osborneclarke.com

Thank you – Q & A