TEAMS 2 & 4THE MICHAEL L. GARGANO 9TH ANNUAL

RESEARCH DAY PRESENTATION

PRESENTERSEDYTA ZYCH & VINNIE MONACO

May 6, 2011Seidenberg School of Computer Science and Information Systems

Pace University, Graduate Center

White Plains, New York

Keystroke Biometric& Stylometry Systems

AGENDA

Team and Project Leader Introductions

KBS & Stylometry Projects Overview

Project Specifications & Deliverables

System Components & Enhancements

Results & Conclusions

Future Work

PROJECT STAKEHOLDERS

Team Members Vinnie Monaco Tyrone Allman Mino Lamrabat Mandar Manohar

Customers / SMEs Dr. Tappert John Stewart Robert Zack

Team Members Edyta Zych Omar Canales Vinnie Monaco Thomas Murphy

Customers / SMEs Dr. Tappert John Stewart

Keystroke Biometric

Stylometry

TWO PROJECTS ACT AS ONE, TWO TEAM LEADS

Person ManagerFacilitate Weekly Meeting ScheduleTask AssignmentsDriving Everyday ActivitiesTech Training & Documentation

Technical ManagerSubject Matter Expert (SME)Technical ScopeDesign & Implementation of all System

EnhancementsProgramming Tasks

OVERVIEW: KEYSTROKE BIOMETRIC SYSTEM

Pace University has conducted over 8 years of research on Keystroke Biometrics

The Keystroke Biometric System (KBS) can be used for both identifying and authenticating users from their typing rhythms

Keystroke dynamics are the patterns of rhythm and timing created when a person types, including: Overall speed Variations of speed moving between specific keys Common errors The length of time that keys are depressed (duration)

This semester’s work focuses solely on the KBS as it relaters to an online test taking environment

OVERVIEW: STYLOMETRY

Stylometry is the study of the unique linguistic styles and writing behaviors of individuals in order to determine authorship

Stylometry uses statistical pattern recognition, and artificial intelligence techniques

Stylometry features typically used to analyze text include word frequencies and identifying patterns in common parts of speech

This semester’s work focuses on text input being used in conjunction with the keystroke analysis to improve authentication results including Determining authorship in documents (Beneficial

academically to assist with on-line test taking) Protecting against plagiarism through a third party

PROJECT SPECIFICATIONS

Work closely with our project customer to define the most appropriate Keystroke & Stylometry Features and add additional features to assist in validating/authenticating the identity of students taking an online exam

Extract the selected Feature Set for Keystroke Biometric and Stylometry Analysis and run experiments to measure program performance utilizing the enhanced systems: Input System, Feature Extractor and Classifier

Run experiments and tests on the data collected to support the identification of subject and online test-taker authorship

PROJECT DELIVERABLES

Systems

User Manuals & Documentation

Website

Presentation

Technical Papers

Input SystemFeature Extractor

Input SystemFeature ExtractorClassifier

KBSStylometry

OVERVIEW OF SYSTEM COMPONENTS

Input System Captures keystroke

and stylometry data in an online test format

Feature Extractor Measures raw data

to obtain a feature vector for each sample

Classifier Uses feature vectors

to test authentication

INPUT SYSTEM ENHANCEMENTS

Upgraded from a Java Applet to a standalone java program.

Implemented a user management system to simulate an online test taking environment

Change to test taking format, instead of free text or copying tasks

Moved to a more general XML data format, to handle both keystroke and stylometry data

More restrictions in place on how users interact with the system Disable cut/copy/paste ability Users must complete the test in full

Capture and log keystrokes from every successful login attempt

FEATURE EXTRACTION ENHANCEMENTS

Feature extraction implemented in the functional language Clojure Easy integration with Java front end Better data handling, filtering, and mapping

capabilities New Normalization method tested

Old formula

New formula Improved outlier removal Integrated stylometry and keystroke features

BENCHMARK RESULTS: 18 SUBJECTS, 180 SAMPLES

Before

After

NORMALIZATION RESULTS ON BENCHMARK DATA

BadGood Still OK

ANALYSIS / RESULTS 40 students, 10 samples each from 1 test Weak training Keystroke and Stylometry biometrics

ANALYSIS / RESULTS 38 students, 20

samples from 2 tests Strong training Stylometry

biometrics

FRR (%)

FAR

(%)

KEYSTROKE COMBINED DATA 38 students, 20 samples each

from 2 tests Weak training ~11% equal error rate

38 students, 20 samples each from 2 tests

2 samples combined yielding 10 samples each

Weak training ~5% equal error rate

FRR (%)

FAR

(%)

FRR (%)

FAR

(%)

0 100

20

0 100

20

KEYSTROKE VS. STYLOMETRY ROC CURVE 38 students, 10 samples from 2 tests Weak training No equal error rate for stylometry

STYLOMETRY COMBINED DATA 40 students, 10 samples

each from 1 test No equal error rate

30 students, 30 samples each from 3 tests

6 samples combined yielding 5 samples each

~30% equal error rate

FRR (%)

FAR

(%)

0 100

60

FRR (%)

FAR

(%)

0100

40

24 STUDENTS, 10 SAMPLES COMBINEDWEAK TRAINING

STYLOMETRY COMBINED DATA

Authenticating students ~32% equal error rate

Authenticating test ~35% equal error rate

FRR (%)

FAR

(%)

0100

100

FRR (%)

FAR

(%)

0100

100

FUTURE WORK

Keystroke and Stylometry Biometrics

Improve stylometry authentication results by identifying important features

Combined more samples to obtain stylometry features on longer text input

Determine if samples may be authenticated to a test, as opposed to the individual

Data Collection Modify the input

system to eliminate some problems with giving an online test Authenticate with

first/last name only Ability to traverse the

questions in the test Integrate keystroke

authentication with users logging into the system

QUESTIONS

TEAMS 2 & 4KEYSTROKE BIOMETRIC

& STYLOMETRY SYSTEMS

THANK YOU!

Tyrone Allman, Omar CanalesMino Lamrabat, Mandar ManoharVinnie Monaco, Thomas Murphy

John Stewart, Dr. Charles TappertRobert Zack, Edyta Zych