Dr. Jay F. Nunamaker, Jr.Director, National Center for Border Security and

Immigration

Regents and Soldwedel Professor ofMIS, Computer Science, and Communication

University of Arizona

Deception Detection Techniques

for Rapid Screening

2National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Why Study Credibility Assessment?

Humans are poor lie-detectors – ~54% accuracy rate for general population– Accuracy deeply affected by base rates– Poor performance affects novices and

professionals

Confidence in judgment is not correlated with accuracy– Affects attentiveness, verification efforts, and

misallocation of resources

26 FEB 2009

3National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Deception Defined

Deception is a message knowingly transmitted with the intent to foster false beliefs or conclusions

Covers common outright deceptions– Fabrications, misdirections, bluffs, etc.

Also covers:– Omissions, concealments, strategic ambiguity,

etc.

26 FEB 2009

4National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Improving Human Detection

Detection accuracy rates may improve with:– Training

Mixed results, but generally significant positive effect

– High stakes scenarios Detection in criminal interviews approaches 80%

accuracy However, the linkage between motivation and

deception performance is currently being debated

– Familiarity with individual’s usual behavior patterns (baseline)

26 FEB 2009

5National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Problems for Humans to Overcome

Truth bias– Tendency to assume all tell truth– Common among lay-people

Othello (lie) bias– Tendency to assume all are lying– Common among law enforcement personnel

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Problems for Humans to Overcome

Excessive focus on nondiscriminant or a small subset of cues– Gaze aversion – Nervous gesturing– Preening

Vigilant observation– Attention required for multiple channels

626 FEB 2009

7National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Border Crossing Environment

26 FEB 2009

8National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Border ScenarioTest Your Skills!

26 FEB 2009

Fluid motion Resonant voice Spontaneous positive affective displays

Results

10National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Border ScenarioTest Your Skills!

26 FEB 2009

Rigid during non-speech, abrupt during speech

Elevated voice frequency Submissive responses

– Low gain– Signs of deference

Negative affective displays– Perhaps in response to agent

Various contextual cues

Results

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Research Purpose

Unobtrusive credibility assessment and intent detection– No sensors attached to the body– Real-time, remote analysis– Scalable and robust for high traffic – Useful across contexts

Investigating technologies to augment/replace the polygraph

1226 FEB 2009

13National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Research Approach

Individual

Inte

rnal

Ext

ern

al

Intent

Exhibited by

Ob

serv

ed P

rofil

es

Pa

st P

rofil

es (

No

rms)

Remotely Observable Cues

Expected Behavioral Norms

GeneralBehavioral

Norms

IndividualBehavioral

Norms

Intent and Deception DetectionSystem

Deviations/Reponses

Observed -vs- Individual norm

Observed -vs- General Norm

Fu

sio

n E

ng

ine

Judgment

Inferred Intent

Deception Detection

- Hostile - Benign

- Deception Indicated- No Deception Indicated

Manifestations of Intent

Behavior Physiology

Physiological Heart Rate Blood Pressure Respiration Iris/Pupil Thermal Blink

Behavioral Linguistic Kinesic Vocalic Eye Tracking Microfacial Chronemic Content

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

15

Non-invasive Credibility Assessment

26 FEB 2009

16National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Approach for Multimodal Credibility Assessment

Validation Multi-institution research program

Multi-method approach– Prototype development– Field studies– Laboratory studies– Surveys

Over 4600 subjects

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

17

Laboratory Layout

Interviewee

Bio TechnicianVideo

Technician

Ca

me

ra 1

Ca

me

ra 2

Camera 3

IR Camera

High Speed

Camera

Thermal Camera

Interviewer

Backdrop

Startle Response Generator

High Speed Camera Station

LDV StationThermal Station

15'-0"

15'

-7 1

1/1

6"

2'-6

"

Interviewee

Ope

rato

r

Eye Tracker

Orientation Station

Debrief Station

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

18

Experiment Lab setup

26 FEB 2009

19

Credibility Assessment Devices(Provided by DACA – Defense Academy for Credibility

Assessment)

26 FEB 2009National Center for Border Security and Immigration – RESEARCH LEAD: University of

Arizona“Challenges and Solutions at the Northern Border“ – Bellevue, WA

LDV THERMAL BLINK

PUPILLOMETRY

EYE-TRACKING

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

20

Laser-Doppler Vibrometry

26 FEB 2009

21National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Laser-Doppler Vibrometry

Uses a laser pointed at the carotid artery Tracks minute changes/movement in the

artery Can remotely track pulse and blood

pressure Currently exploring other identifiable

physiological features– Respiration– Muscle tension– Artery stiffness

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

22

Laser-Doppler VibrometryOperational Benefits and Drawbacks

Benefits For use with screening

questions– No narrative responses

Tracks well known physiological features

Drawbacks Noise introduced by

speaking, movement, etc.

Small target area

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

23

Thermal Video Imaging Technology

26 FEB 2009

24National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Thermal Video Imaging Technology

Uses a thermal camera

Tracks minute changes in skin temperature

Can identify increase blood flow to the brain

Arousal signature shows as a temperature plume in the lower forehead and between the eyes

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

25

Thermal Video Imaging Technology

Operational Benefits and DrawbacksBenefits For use with screening

questions

Drawbacks Expensive Noisy equipment Time intensive

calibration Sensitive to

environmental changes

Data intensive

26 FEB 2009

26National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

High-speed Blink Imaging Technology

26 FEB 2009

27National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

High-speed Blink Imaging Technology

Uses a high speed camera (~250 fps) Tracks blink responses to questions

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

28

High-speed Blink ImagingOperational Benefits and Drawbacks

Benefits For use with screening

questions

Drawbacks Used with a startle

generator– Can be unwieldy during

an interview

Data intensive

26 FEB 2009

29National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Pupillometric Video Imaging Technology

26 FEB 2009

30National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Pupillometric Imaging Technology

Uses an IR camera to track pupil dilation Deceptive responses correlate with pupil

dilation

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

31

Pupillometric Imaging Technology

Operational Benefits and DrawbacksBenefits For use with screening

questions Brief responses need

be measured

Drawbacks Sensitive to noise Most diagnostic when

subject is stationary Requires unobstructed

view of eyes

26 FEB 2009

32National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

26 FEB 2009

Eye-tracking Systems

Uses multiple IR cameras Tracks search patterns as participant views

an image Can separate familiar images from

unfamiliar images

33National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

Eye-tracking Systems & Applications

26 FEB 2009

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

34

Eye-tracking SystemsOperational Benefits and Drawbacks

Benefits Deployable system Non threatening

interaction Immediate feedback

Drawbacks Requires guilty

knowledge Requires subject

cooperation– Calibration– Proper scanning of

images Tracking not possible

with all subjects

26 FEB 2009

Embodied Avatar KioskRapid Deception Assessment Screening

Mission– Rapid, automated deception assessments using

kiosk-based embodied avatars and intelligent agents

Objectives– Identify deception cues in rapid standing

assessments – Intelligent agent-based real-time cue processing

& analysis– Temporal cue pattern identification– Embodied avatar effectiveness optimization– Repeated assessment analysis and baseline

profiling– Effective threat index and deception index

development

26 FEB 2009National Center for Border Security and Immigration – RESEARCH LEAD: University of

Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA 35

Intelligent Agent

(IA) Selects

Interview

Embodied Avatar presents Question

Sensors CaptureSubject

Responses

Real-Time Response

/ Cue Analysis

IA Fuses Analyses, creates ranking

IASelects

Next Question

IA Generates

Final Ranking and

Analysis

Done?

Embodied Avatar KioskRapid Deception Assessment Screening

National Center for Border Security and Immigration – RESEARCH LEAD: University of Arizona “Challenges and Solutions at the Northern Border“ – Bellevue, WA

37

Embodied Avatar KioskVideo Example

26 FEB 2009