vision extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/schulz.pdf · kyle...

Gaze Detec)on and Tracking Kyle Schulz • CSCI 512 • Spring 2013 Vision Exten)on

Upload: others

Post on 22-Aug-2020

1 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

VisionExten)on

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

Goals

1.  Enhanceuser’sexperience• Extendvisionbeyondrestric@ons• Createseamlessintegra@on

2.  Enhanceviewer’sexperience• Createillusionofreality• Mimicfinernuances

Page 3: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

Process

1.  Buildcapturemechanism2.  Captureuser’seye3.  Locateeyesinreal@me4.  Determinegaze5.  Projectontoexternalcameras6.  Compositexternals

CurrentProgress

Page 4: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

BuildingtheCaptureMechanism

CheapWebcam

IRFilter

CameraFilm

Page 5: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

BuildingtheCaptureMechanism

Lens

USBPower

IRLEDs

Page 6: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

CaptureUser’sEye

•  OpenCV•  C++•  Grabsimagefromvideo

stream•  Issomewhatclunkywith

mul@plecameras•  IRallowsforsemi‐stable

imagecondi@ons

Page 7: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

LocateEyesinReal)me

•  Converttograyscale•  Dialateanderodetoremove

LEDglareandnoise•  Erodeanddialatetoremove

damagedCCDpixelsanddust

•  Playwithhistogramstoenhancecontrast

Page 8: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

LocateEyesinReal)me

•  DynamicThresholding•  Storesaminandmax

successfulthreshold•  Checksmin,maxand

averagewhenlost•  Returnsanareasimilartoan

elipse•  LessaccurateinIRsitua@ons

Page 9: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

LocateEyesinReal)me

•  Applygausianblur•  Usecannyeedgedetector•  Findthe“blobs”,compute

theirarea•  Findboundingrectangle•  Compareareawithbounding

rectangle’sellipsearea•  Returnbestcandidate

Page 10: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

LocateEyesinReal)me

•  Useeitherorbothsolu@onstobestdeterminepupilloca@on

•  Persistance,smoothing

Page 11: Vision Extenon - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/Schulz.pdf · Kyle Schulz • CSCI 512 • Spring 2013 Goals 1. Enhance user’s experience • Extend

GazeDetec)onandTrackingKyleSchulz•CSCI512•Spring2013

DetermineGaze

PossibleSolu)ons

•  Usewidthofirisandloca@onofcamera

•  Calibratewithextentsofvision

•  Detecteyeasawhole

Denver Defenders - cs-courses.mines.educs-courses.mines.edu/csci370/FS2015/FinRpt_TGC.pdf · LibGDX is the free graphics library we are using to create this game at the request of

Rubik’s Cube Recognition and Identification using Computer ...cs-courses.mines.edu/csci507/projects/2016/Appapogu_Saichand.pdf · Introduction Named after Erno Rubik, its inventor

Mamluk Egyptian Copper Coinage Before 759/1357-1358: A ...mamluk.uchicago.edu/MSR_V_2001-Schulz.pdf · MAMLU±K STUDIES REVIEW VOL. 5, 2001 27 Copper coins are also said to have fulfilled

COLORADO SCHOOL OF MINES - cs-courses.mines.educs-courses.mines.edu/csci370/FS2015/FinRpt_CSMMehta.pdf · COLORADO SCHOOL OF MINES AlgoBOWL by Arnaud Filliat, Justin Frazer, Robyn

Fowler Software Design - cs-courses.mines.educs-courses.mines.edu/csci370/FS2009/FowlerFinal.pdf · legacy FoxPro and SQL programming languages. ... 2. User clicks “Print Report”

Computer Vision - cs-courses.mines.educs-courses.mines.edu/csci507/schedule/06/3Dto3DTransforms.pdf · Computer Vision Rotations in 3D 4 • The rotation matrix has 9 elements •

Interactive Design Space Exploration and Optimization for CAD …people.csail.mit.edu/aschulz/optCAD/a11-schulz.pdf · 2017-07-24 · Interactive Design Space Exploration and Optimization

AUGMENTED REALITY VIDEO PLAYLISTcs-courses.mines.edu/csci507/projects/2015/Chandra.pdf · AUGMENTED REALITY VIDEO PLAYLIST Surya Chandra EENG512 . 1/29 Augmented Reality ... POINT

Pivotal Tracker Kanban Prototype - cs-courses.mines.educs-courses.mines.edu/csci370/FS2017/FinalReport/Pivotal.pdf · Pivotal Tracker is a project management application designed

LANDSCAPE AND DISEASES. THE MIDDLE HOLOCENE SAHARA: …riscurisicatastrofe.reviste.ubbcluj.ro/.../Schulz.pdf · Interrelationships and interdependencies within the traingle: man-landscape-

Project ADD - cs-courses.mines.educs-courses.mines.edu/csci370/FS2008/CSM5Final.doc · Web viewProject ADD. CSM#5 Software Engineering Java Tutorial Plug-in . Client Report. Zachary

Microsoft Visual Studio Tutorialcs-courses.mines.edu/csci507/schedule/13/VS_C++.pdf · Microsoft Visual Studio Tutorial Colorado School of Mines Department of Electrical Engineering

harles Monroe Schulz (born 1922) Schulz.pdf · overcame the dire times and his own lack of education with metronome work habits." Dena encouraged Charles's drawing but died from cancer

Augmented Reality - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2013/French_Augmented Rea… · What is Augmented Reality (AR)? A live view of a real, physical environment

Team Medecipher Final Report - cs-courses.mines.educs-courses.mines.edu/csci370/FS2020S/FinalReport/Medecipher.pdf · Final Website Mockup Model Description: Schedule module: This

Computer Visioncs-courses.mines.edu/csci507/schedule/27/Ransac.pdfAlgorithm (RANSAC) 1) Randomly select a minimal set of points and solve for the transformation 2) Count the number

Xbusi Missing Piece - Computer Sciencecs-courses.mines.edu/csci507/projects/2013/PuzzleSolver_Presentat… · ComparingEdges 8 Firstand+Last Used+Matches+from+Angry+ Birds+Puzzle

Spatial Language for Mobile Robots: The Formation and ...staff.itee.uq.edu.au/janetw/papers/PhD 2008 Schulz.pdf · Janet Wiles and Gordon Wyeth put together the initial proposal for

u.s.c. - Minnesotamn.gov/commerce-stat/pdfs/superseding-indictment-schulz.pdf · U.S. v. Angela April Schulz, et al. Criminal No. 16-341 (MJD/FLN) kickbacks to runners, including

Human Robot Interface - cs-courses.mines.educs-courses.mines.edu/csci370/FS2016/FinRpt_Zhang1.pdf · ROS Nodes A ROS application is rarely a single program in the traditional sense

Water Hero - cs-courses.mines.educs-courses.mines.edu/csci370/FS2019S/FinalReport/... · Storyboard Figure 2 - Storyboard The storyboard (Figure 2) shows the navigation between the

DRACULA - cs-courses.mines.educs-courses.mines.edu/csci370/FS2015/FinRpt_CSMTurner.pdf · known as “Critical Functions,” to score different designs, referred to as “Analogies,”

Computer Visioncs-courses.mines.edu/csci507/schedule/09/TemplateMatching.pdf · Colorado School of Mines Computer Vision Cross Correlation • Expanding the expression for SSD: •

Polyhedral Approaches to Machine Schedulingweb.mit.edu/schulz/www/epapers/queyranne-schulz.pdf · 2008. 8. 13. · con vex hull of these points Ð particularly in order to obtain

Hand Gesture Recognition - Computer Sciencecs-courses.mines.edu/csci507/projects/2014/Jonathan... · • Fingertip Detection using MATLAB image processing toolbox. • Combination

Martin Schulz University of Washington to appear in ...healdric/Classes/Soci245/Schulz.pdf · Martin Schulz University of Washington to appear in: ... have left the organization

“Agricola” Board Game Assist Program EGGN 512 – Computer ...cs-courses.mines.edu/csci507/projects/2012/Agricola... · Color images are taken using a DSLR camera as JPEGs under

Computer Visioncs-courses.mines.edu/csci507/schedule/35/Tracking.pdf · Colorado School of Mines Computer Vision Tracking Patch Features •We want to select good features to track

Computer Visioncs-courses.mines.edu/csci507/schedule/11/BinaryImages.pdfColorado School of Mines Computer Vision Concentric contrasting circle (CCC) target • The target is a white

Eye Feature Detection - cs-courses.mines.educs-courses.mines.edu/csci507/projects/2014/FinalProject---Eye Featu… · CONTENT ‣1 Introduction ‣ 1.1 Motivation ‣ 1.2 Goals ‣

Model evaluation through AeroCom Michael Schulz Jan ...aerocom.met.no/DATA/AEROCOM_PUBLIC/reykjavik/schulz.pdf · Model evaluation through AeroCom Michael Schulz Jan Griesfeller,

Automatic generation and detection of highly reliable fiducial …cs-courses.mines.edu/csci507/schedule/24/ArUco.pdf · 2016-10-13 · Automatic generation and detection of highly

3 - Michael Schulz.pdf · Created Date: 4/23/2011 11:13:58 AM

Object Recognition for Amazon Picking Challenge Patrick Greenecs-courses.mines.edu/csci507/projects/2015/Greene.pdf · Object Recognition for Amazon Picking Challenge Patrick Greene

Computer Visioncs-courses.mines.edu/csci507/schedule/34/Pattern...% meas(150,4) - each row is a pattern (a 4-dimensional vector) % species{150} - each element is the name of a flower